• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Validity of Research

Validity of Research



Concepts of research methods briefly presented, with embedded links to some videos.

Concepts of research methods briefly presented, with embedded links to some videos.



Total Views
Views on SlideShare
Embed Views



1 Embed 6

https://courses.css.edu 6



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Validity of Research Validity of Research Presentation Transcript

    • PSY 3330 Research Methods
      Research Methods:Validity and Threats to Validity
    • Generating Research Hypotheses(Review – Figure 1.2)
      Ideas lead to
      library research
      Initial theory
      Predictionsor hypotheses
      Operationalized in new studies
      Findings used to support, refute or refine theory.
    • Characteristics of a Good Study Question
      F Feasible
      I Interesting
      N Novel
      E Ethical
      R Relevant
    • Actually testing three sets of hypotheses
      The null hypothesis
      The confounding variable hypotheses
      The causal hypothesis
      Accept causal hypothesis only if you
      reject null hypothesis (statistical analysis)
      rule out each potential confounding variable hypothesis (based on appropriate controls)
      Testing Research Hypotheses
    • Criteria for Nomothetic Causality (See Table 2.2 on p 34)
      Correlation (also called covariation)
      Relationship found between variables
      Time order
      Cause must occur before result
      Alternative explanations must be eliminated from possibility
      Experiments are intended to reduce or rule out alternative explanations and confoundingvariables
    • Asking “the Question”
      The PICO format:
      P Population
      I Intervention or Interest area
      C Comparison intervention or status
      O Outcome
    • Applying the “PICO” format
      Original: “What is the usefulness or accuracy of the current 1-10 pain scale assessment in treating a patient’s pain, and what are other options that may prove more useful?”
      Research: Does a 10 point pain Visual Analog Scale (____, ____) accurately assess pain in the first day postop abdominal total hysterectomy patient when compared with the Faces Pain Scale (Pasaro, 1997)?
    • After the experiment, what do you think about the null hypothesis?
      What do you think about the research (causal) hypothesis?
      Evaluating Hypotheses
    • Statistical Validity – carrying out the actual statistical analysis properly
      Construct Validityrefer most often to a characteristic of an instrument but also to the whole study
      Internal Validity refers to a characteristic of a study’s design
      External Validity refers to the generalizability of study findings
      Types of Research Validity
    • Statistical Validity
      Do the variables covary?
      Is the relationship statistically significant?
      Threatened by
      Unreliable measures
      Violations of statistical assumptions
      How do we detect these problems???
      Strengthened by
      Using well validated measures
      Having approximately equal sample sizes in each group)
    • Construct Validity
      Can we infer higher-0rder (more abstract) constructs from the operations used in our experiment or study?
      E.g., Milgram’s study – authority? Social conformity?
      Is our theory the best explanation for the results?
      Threatened by
      Any alternative explanation for the results
      HOW do we locate these alternative explanations?
      Strengthened by
      Using well-validated constructs to build the theoretical predictions for the study
      See list of threats on p. 171 in Table 6.2
    • Do the results apply to the broader population?
      Threatened by
      Unrepresentative samples
      Generalizing beyond the limits of the sample
      HOW do we know when this problem is present???
      Strengthened by
      Gathering a representative sample (if possible)
      Clearly describing sample, so that other researchers will know the limits of generalization
      External Validity
    • Is the independent variable responsible for theobserved changes in the dependent variable?
      Threatened by
      Confounding variables
      HOW do we detect the presence of confounding variables????
      Strengthened by
      Adding adequate controls to reduce or eliminate confounding
      Internal Validity
    • Confounding and internal validity
      Many sources for confounding (covered next)
      With proper controls, confounding can be virtually eliminated (see Chapter 9)
      Confounding and construct validity
      Make sure that you have considered alternative theoretical explanations for the anticipated phenomenon
      Avoiding Confounding
    • Pretest-Posttest Research Design
      Single-group, pretest-posttest design compares pre-treatment and post-treatment scores to determine improvement
      Fails to control most sources of confounding
    • Assumptions threaten validityDan Ariely – Predictably Irrational (TED)
      Many theories include an assumption about human motivations or behavior
      Behavioral economists & some psychologists explore deeply whether these assumptions prove accurate
      See Dan Ariely’s website, where he often has experiments.
    • Sources of Internal Invalidity
      Historicalevents may occur during the course of the experiment.
      Remember Pygmalion effect & its story
      Maturationof the subjects.
      Testingand retesting can influence awareness of variables or behavior
      Learn Hawthorne effect & its story.
      Instrumentation– measurement methods or procedures may not be equivalent
    • Sources of Internal Invalidity
      Statistical regression of subjects starting out in extreme positions.
      Selection biases (we will see several types)
      Experimental mortality(a.k.a. attrition) – subjects drop out of the study before it's finished.
      Sequence effects – Performance on one measure is related to previous experience with other measures. Outcome depends on the sequence of measures.
    • Social threats to validity
      Demoralization subjects incontrol group find out, loseinterest in study, stop trying
      Diffusionof treatment (those who get the experimental stimulus spread it to controls)
      Rivalry(controls change behavior to try to beat the experimental group)
      Equalizationof treatment (researcher compensates controls for not getting treatment)
    • Social Psychology Experiment:Conformity to Norms(file)
      Experiment:Conformity to gender roles. We placed common male and female signs on opposite transparent doors instructing people to walk through the correct entrance. (Sorry, no music. Youtube made me remove it)
    • Participants are not passive
      They try to understand the study to help them to know what they “should do” (termed subject effects)
      Respond to subtle cues about what is expected (termed demand characteristics)
      Placebo effect: treatment effect due to expectations that the treatment will work
      Subject Effects
    • Based on the expectations of the researcher
      Can affect the outcome of studies if not controlled
      May be due to the experimenter providing demand characteristics to the participant
      Not the same as scientific fraud (which is deliberate)
      Experimenter Effects
    • Validity, Control, and Constraint
      Three closely-tied concepts
      The accuracy of the study or procedure
      Increased by using appropriate control procedures
      The more controls we employ, the higher the level of constraint
      Controls may increase some types of validity while, by their unnatural aspect, decreasing other types of validity.
    • Risk is balanced by reward
      A poorly designed study will provide no useful information; therefore, any risk would be unacceptable
      Informed Consent
      Virtually guarantees that you will have confounding due to selection because some people will refuse to participate
      A small price to pay to maintain ethical standards
      Ethical Principles
    • Start by building a research hypothesis
      Testing the research hypothesis is actually testing three hypotheses
      (1) null; (2) confounding-variable; (3) causal
      Several types of validity
      Many potential confounding variables
      Subject and experimenter effects can also affect the outcome of the study
    • PSY 3330 Research Methods
      Research Methods:Validity and Threats to Validity