Research Theory, Design, and Methods Walden University Threats to Internal Validity Threats to Internal Validity (Shadish, Cook & Campbell, 2002) 1. Ambiguous temporal precedence. Based on the design, unable to determine with certainty which variable occurred first or which variable caused the other. Thus, unable to conclude with certainty cause-effect relationship. Correlation of two variables does not prove causation. 2. Selection. The procedures for selecting participants (e.g., self-selection or researcher sampling and assignment procedures) result in systematic differences across conditions (e.g., experimental-control). Thus, unable to conclude with certainty that the “intervention” caused the effect; could be due to way in which participants are selected. 3. History. Other events occur during the course of treatment that can interfere with treatment effects, and could account for outcomes. Thus, unable to conclude with certainty that the “intervention” caused the effect; could be due to some other event to which the participants were exposed. 4. Maturation. Natural changes that participants experience (e.g., grow older, get tired) during the course of the intervention could account for the outcomes. Thus, unable to conclude with certainty that the “intervention” caused the effect; could be due to the natural change/maturation of the participants. 5. Regressionartifacts. Participants who are at extreme ends of the measure (score higher or lower than average) are likely to “regress” toward the mean (scores get lower or higher, respectively) on other measures or retest on same measure. Thus, regression can be confused with treatment effect. 6. Attrition (mortality). Refers to drop out or failure to complete the treatment/study activities. If differential drop out across groups (e.g., experimental-control) occurs, could confound the results. Thus, effects may be due to drop out rather than treatment. 7. Testing. Experience with test/measure influences scores on retest. For example, familiarity with testing procedures, practice effects, or reactivity can influence subsequent performance on the same test. 8. Instrumentation. The measure changes over time (e.g., from pretest to posttest) thus making it difficult to determine if effects or outcomes are due to instrument vs. treatment. For example, observers change definitions of behaviors they are tracking, or the researcher alters administration of test items from pretest to posttest. 9. Additive and interactive effects of threats to validity. Single threats interact, such that the occurrence of multiple threats has an additive effect. For example, selection can interact with history, maturation, or instrumentation. Reference Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton- Mifflin. 1 of 2 Research Theory, Design, and Methods Walden University Measurement of Variables On ...

1. Research Theory, Design, and Methods
Walden University
Threats to Internal Validity
Threats to Internal Validity
(Shadish, Cook & Campbell, 2002)
1. Ambiguous temporal precedence. Based on the design, unable
to determine with certainty which variable occurred first or
which variable caused the other. Thus, unable to conclude with
certainty cause-effect relationship. Correlation of two variables
does not prove causation.
2. Selection. The procedures for selecting participants (e.g.,
self-selection or researcher sampling and assignment
procedures) result in systematic differences across conditions
(e.g., experimental-control). Thus, unable to conclude with
certainty that the “intervention” caused the effect; could be due
to way in which participants are selected.
3. History. Other events occur during the course of treatment
that can interfere with treatment effects, and could account for
outcomes. Thus, unable to conclude with certainty that the
“intervention” caused the effect; could be due to some other
event to which the participants were exposed.
4. Maturation. Natural changes that participants experience
(e.g., grow older, get tired) during the course of the intervention
could account for the outcomes. Thus, unable to conclude with
certainty that the “intervention” caused the effect; could be due
to the natural change/maturation of the participants.

2. 5. Regressionartifacts. Participants who are at extreme ends of
the measure (score higher or lower than average) are likely to
“regress” toward the mean (scores get lower or higher,
respectively) on other measures or retest on same measure.
Thus, regression can be confused with treatment effect.
6. Attrition (mortality). Refers to drop out or failure to
complete the treatment/study activities. If differential drop out
across groups (e.g., experimental-control) occurs, could
confound the results. Thus, effects may be due to drop out
rather than treatment.
7. Testing. Experience with test/measure influences scores on
retest. For example, familiarity with testing procedures, practice
effects, or reactivity can influence subsequent performance on
the same test.
8. Instrumentation. The measure changes over time (e.g., from
pretest to posttest) thus making it difficult to determine if
effects or outcomes are due to instrument vs. treatment. For
example, observers change definitions of behaviors they are
tracking, or the researcher alters administration of test items
from pretest to posttest.
9. Additive and interactive effects of threats to validity. Single
threats interact, such that the occurrence of multiple threats has
an additive effect. For example, selection can interact with
history, maturation, or instrumentation.
Reference
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002).
Experimental and quasi-experimental designs for generalized
causal inference. Boston: Houghton- Mifflin.
1 of 2

3. Research Theory, Design, and Methods
Walden University
Measurement of Variables
One of the important aspects of conducting quantitative
research is deciding how you will measure your variables. If
you are not clear about what a variable is (e.g., gender) as
opposed to a value of a variable (e.g., female, male) or the
difference between an operational definition of a variable (i.e.,
how you will measure and code a variable) and a conceptual
definition of a variable (i.e., an explanation of the construct in
plain English) please see chapters 5 and 6 in Babbie (2002). If
you do not know the different types of measurement reliability
(e.g., test-retest, internal consistency), measurement validity
(e.g., predictive, construct), or levels of measurement (i.e.,
nominal, ,ordinal, interval, and ratio) please see chapters 1- 3 in
Walsh and Betz (2001) or chapters 1-5 in Thorndike (2005).
The first three chapters in Walsh and Betz (2001) cover 144
pages and do a good job of covering the basics of measurement
including levels of measurement (nominal, ordinal, interval, and
ratio), reliability (e.g., test-retest, internal consistency), validity
(e.g., predictive, concurrent), types of scores (e.g., standard
scores, percentiles), norms, and cultural issues. Chapters 1 - 5
of Thorndike cover 217 pages and go into more detail than
Walsh and Betz. In addition, it includes other chapters on
measuring attitudes and using rating scales (Chapter 12) and
instrument development (Chapter 15).
You do not have to become an expert on these topics for the
assignments in this course; however, you will need to know the
basics when you work on the assignments that involve
quantitative research.

4. References
Babbie, E. (2002). The basics of social research (2nd ed.).
Belmont, CA: Wadsworth.
Thorndike, R.M. (2005). Measurement and evaluation in
psychology and education. Upper Saddle River, NJ: Pearson.
Walsh, W.B. & Betz, N.E. (2001). Tests and assessment (4th
ed.). Upper Saddle River, NJ: Prentice-Hall.
1 of 2
Research Theory, Design, and Methods
Walden University
Research Problems
Applied Management and Decision Sciences
Organizations are human systems. As such, each member of the
system brings to bear the full range of psychological and
emotional behavior while also applying themselves to their
work which is highly prescriptive and ostensibly rational.
Because of this, human systems experience high levels of
inefficiency as emotions, personality, and unconscious impulses
interfere with the objectivity and logical processes at work. The
question is, to what extent do individuals' idiosyncrasies
interfere with the expectations of their performance? A
secondary question is, are there managerial tools to mitigate
these forces among employees while they are in the workplace?
Counseling Education
When a parent or guardian is incarcerated, that incarceration
has an impact on family members, especially children, and the
relationship that the incarcerated individual has with family

5. members. As human service professionals, we are concerned
particularly with the family experience for ex-offenders and
with designing interventions that improve the relationship
between ex-offenders and their children and interventions that
reduce the likelihood of recidivism with a formerly-incarcerated
individual. However, in order to design interventions to address
the latter, we must first understand the internal and external
factors that result in ex-offenders committing crimes again. The
question that we are interested in answering is what are the
external and internal factors that contribute to recidivism and
what interventions are effective in minimizing the impact of
those factors.
Education
The impact of No Child Left Behind (NCLB) is being felt at all
levels of education. Even though a new administration has
continued and redoubled the goals of NCLB, no research has yet
been done on the effect/impact NCLB has had on students'
motivation to learn at each educational level—from Early
Childhood through Higher Education and Adult Learning, where
the impacts of NCLB on recent graduates are only beginning to
be felt—and in each educational venue, from traditional public
education to alternative programs and private educational
environments. (Students can tailor their designs to their own
areas of specialization.)
Health Services and Public Health
Health disparities are a growing concern in the U.S. There are
also a significant proportion of people with limited or no health
insurance which may prevent them from receiving adequate
health care, medication, and the use preventative measures such
as exercise and diet. How would you address this issue?
Public Policy and Administration
Elected and appointed officials at all levels of government are
faced with the dilemma of raising demands for services and

6. diminishing financial resources needed to meet those demands.
The traditional method of using a public agency to implement
policy and deliver services is no longer possible or even
desirable in many cases. Governments in the United States and
throughout the world have formed partnerships with nonprofit
or nongovernmental organizations as a means to implement
policies and deliver services. In many cases, the new
partnerships have placed administrators from two very different
organizational cultures in direct contact with one another on a
daily basis. Public and nonprofit administrators must learn to
accommodate the organizational culture of their counterparts if
the new partnerships are to succeed. Scholars studying in the
fields of public and nonprofit administration can provide a great
service to their counterparts in practice by addressing the
research problem of how to integrate the organizational cultures
of their two sectors.
PAGE
2