#10 Location In The Research Process Formulating Research Questions Previous Research Craft Knowledge Appropriate Design A...
Internal and External Validity <ul><li>Internal Validity </li></ul><ul><ul><li>The extent to which alternate explanations,...
Internal Validity Threats <ul><li>History </li></ul><ul><ul><li>Life Events, within and across participants </li></ul></ul...
<ul><li>Selection Biases </li></ul><ul><ul><li>Initial differences in groups, based on how they are selected </li></ul></u...
First Question <ul><li>Administrators at Spokane Mental Health: </li></ul><ul><ul><li>Have therapists administer a reliabl...
External Validity Threats <ul><li>Generalizability Across Subjects </li></ul><ul><ul><li>How well will these results gener...
<ul><li>Reactive Arrangements </li></ul><ul><ul><li>Influence of being aware that you're in a study &quot;Hawthorne Effect...
Second Question <ul><li>The Eastern Region DCFS director wants to know if family preservation really works, so he </li></u...
Criteria for Inferring Causality <ul><li>Temporal Ordering </li></ul><ul><ul><li>The independent (causal) variable precede...
Pre-Experimental Designs   <ul><li>These designs rank low for accepted evidence based practice research due to the very lo...
<ul><li>Posttest Only With Nonequivalent Groups </li></ul><ul><li>(Static-Group comparison design) X O </li></ul><ul><li>O...
Nonequivalent Control Groups Designs   <ul><li>O  X  O </li></ul><ul><li>O  O </li></ul><ul><li>Use existing groups, which...
Essential Components of Experimental Designs <ul><li>1. Random assignment of participants to experimental and control grou...
Posttest Only Control Group Designs <ul><li>R  X  O </li></ul><ul><li>R  O </li></ul><ul><li>Randomly assign participants ...
Repeated Measures Control Group Designs  (pretest-posttest) <ul><li>R O X O </li></ul><ul><li>R O O </li></ul><ul><li>Rand...
Other Experimental Designs <ul><li>Solomon four-group design </li></ul><ul><li>R 0 X O </li></ul><ul><li>R O O </li></ul><...
Third Question <ul><li>A researcher at Spokane Mental Health </li></ul><ul><ul><li>Has parents of autistic children comple...
Upcoming SlideShare
Loading in …5
×

Lecture 10 between groups designs

2,705 views

Published on

Published in: Health & Medicine, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,705
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
89
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Lecture 10 between groups designs

  1. 1. #10 Location In The Research Process Formulating Research Questions Previous Research Craft Knowledge Appropriate Design Appropriate Sampling Decisions Ensure Research Ethics Data Collection & Management Data Analysis Interpretation Dissemination Ongoing Research
  2. 2. Internal and External Validity <ul><li>Internal Validity </li></ul><ul><ul><li>The extent to which alternate explanations, beyond the variable of interest, can be ruled out </li></ul></ul><ul><li>External Validity </li></ul><ul><ul><li>The extent to which results of an experiment can be generalized beyond the conditions of an experiment </li></ul></ul>
  3. 3. Internal Validity Threats <ul><li>History </li></ul><ul><ul><li>Life Events, within and across participants </li></ul></ul><ul><li>Maturation </li></ul><ul><ul><li>Internal change unrelated to variable </li></ul></ul><ul><li>Testing </li></ul><ul><ul><li>How repeated measurement effects performance </li></ul></ul><ul><li>Instrumentation </li></ul><ul><ul><li>Instability in the measuring tool over time </li></ul></ul><ul><li>Statistical Regression </li></ul><ul><ul><li>The movement of scores, especially extreme scores, towards the mean over repeated measures </li></ul></ul>
  4. 4. <ul><li>Selection Biases </li></ul><ul><ul><li>Initial differences in groups, based on how they are selected </li></ul></ul><ul><li>Attrition </li></ul><ul><ul><li>Subjects who leave are different than subjects who stay </li></ul></ul><ul><li>Ambiguity about the direction of causal influence </li></ul><ul><ul><li>When we can not determine the time order of the relationship </li></ul></ul>Internal Validity Threats
  5. 5. First Question <ul><li>Administrators at Spokane Mental Health: </li></ul><ul><ul><li>Have therapists administer a reliable and valid depression inventory to clients who are diagnosed with depression at intake </li></ul></ul><ul><ul><li>This occurs during the first and last therapy session </li></ul></ul><ul><ul><li>About one-third of the clients do not complete a termination test </li></ul></ul><ul><ul><li>They compare the pretest and posttest scores to see if therapy is effective for depression </li></ul></ul><ul><li>Identify at least two internal validity threats that are present within this research method </li></ul>
  6. 6. External Validity Threats <ul><li>Generalizability Across Subjects </li></ul><ul><ul><li>How well will these results generalize to people who are different from those in the study </li></ul></ul><ul><li>Generalizability Across Settings </li></ul><ul><ul><li>The extent to which results apply in other situations </li></ul></ul><ul><li>Generalizability Across Measures </li></ul><ul><ul><li>If response was measured differently how would results generalize </li></ul></ul><ul><li>Generalizability Across Times </li></ul><ul><ul><li>Would response differ across time of day, of week </li></ul></ul><ul><li>Generalizability Across Change Agents </li></ul><ul><ul><li>How well do findings apply to different teachers, therapists, parents </li></ul></ul>
  7. 7. <ul><li>Reactive Arrangements </li></ul><ul><ul><li>Influence of being aware that you're in a study &quot;Hawthorne Effect&quot; </li></ul></ul><ul><li>Reactive Measurements </li></ul><ul><ul><li>Influence of measurement or assessment on response </li></ul></ul><ul><li>Pretest Sensitization </li></ul><ul><ul><li>Pretest may influence behavior in study, set a demand characteristic </li></ul></ul><ul><li>Multiple Treatment Inference </li></ul><ul><ul><li>Were the participants exposed to multiple treatments, was it an interaction of treatments that led to the results </li></ul></ul>External Validity Threats
  8. 8. Second Question <ul><li>The Eastern Region DCFS director wants to know if family preservation really works, so he </li></ul><ul><ul><li>Identifies all of the cases referred to family preservation, and those whose children were placed in foster care, during calendar year 2001 </li></ul></ul><ul><ul><li>Uses the DCFS database to track time to permanency for these cases </li></ul></ul><ul><ul><li>Compares the family preservation and foster care cases on time to permanency </li></ul></ul><ul><li>Identify at least two external validity threats that are present within your sites research problem </li></ul>
  9. 9. Criteria for Inferring Causality <ul><li>Temporal Ordering </li></ul><ul><ul><li>The independent (causal) variable precedes the dependent variable in time </li></ul></ul><ul><li>Empirical Correlation </li></ul><ul><ul><li>The independent and dependent variables be strongly correlated with each other </li></ul></ul><ul><li>Internal Validity </li></ul><ul><ul><li>The relationship between the independent and dependent variables cannot be explained by the functioning of a third variable </li></ul></ul>
  10. 10. Pre-Experimental Designs <ul><li>These designs rank low for accepted evidence based practice research due to the very low degree of internal validity </li></ul><ul><li>Uncontrolled Case Studies X O </li></ul><ul><ul><li>Simply makes an observation after a stimulus is presented </li></ul></ul><ul><ul><li>Address temporal ordering (no threats to internal validity are controlled) </li></ul></ul><ul><ul><li>No correlation addressed </li></ul></ul><ul><li>One Group Repeated Measures O X O </li></ul><ul><ul><li>Makes observations of one group of subjects before and after a stimulus is presented </li></ul></ul><ul><ul><li>Address temporal ordering and empirical correlation </li></ul></ul>
  11. 11. <ul><li>Posttest Only With Nonequivalent Groups </li></ul><ul><li>(Static-Group comparison design) X O </li></ul><ul><li>O </li></ul><ul><ul><li>makes observations of two groups, one who receives the stimulus and one that does not, following the stimulus presentation </li></ul></ul><ul><ul><li>groups are not necessarily equivalent since there is no random assignment to groups </li></ul></ul><ul><ul><li>address empirical correlation and temporal ordering </li></ul></ul>Pre-Experimental Designs
  12. 12. Nonequivalent Control Groups Designs <ul><li>O X O </li></ul><ul><li>O O </li></ul><ul><li>Use existing groups, which are not randomly assigned to conditions </li></ul><ul><li>Collect pretest data on both groups </li></ul><ul><li>Apply the independent variable to one group but not the other </li></ul><ul><li>Collect posttest data on both groups </li></ul><ul><li>Address temporal ordering and empirical correlation but not internal validity </li></ul>
  13. 13. Essential Components of Experimental Designs <ul><li>1. Random assignment of participants to experimental and control groups (not to be confused with random sampling) </li></ul><ul><li>Randomization refers to participants of a study being randomly assigned to control and experimental groups; done to increase internal validity </li></ul><ul><li>Random Sampling refers to participants of a study being randomly drawn from an population, theoretically being representative of that population; done to increase external validity (generalizability) </li></ul><ul><li>2. Applying the independent variable to the experimental group but not the control group </li></ul><ul><li>3. Comparing the experimental and control groups </li></ul><ul><li>Experimental designs allow us to examine the direct, causal effect of one variable on another and to make unambiguous inferences. </li></ul>
  14. 14. Posttest Only Control Group Designs <ul><li>R X O </li></ul><ul><li>R O </li></ul><ul><li>Randomly assign participants to conditions </li></ul><ul><li>Apply the independent variable to the experimental group but not the control group </li></ul><ul><li>Collect posttest data on both groups </li></ul>
  15. 15. Repeated Measures Control Group Designs (pretest-posttest) <ul><li>R O X O </li></ul><ul><li>R O O </li></ul><ul><li>Randomly assign participants to conditions </li></ul><ul><li>Collect pretest data on both groups </li></ul><ul><li>Apply the independent variable to the experimental group but not the control group </li></ul><ul><li>Collect posttest data on both groups </li></ul>
  16. 16. Other Experimental Designs <ul><li>Solomon four-group design </li></ul><ul><li>R 0 X O </li></ul><ul><li>R O O </li></ul><ul><li>R X O </li></ul><ul><li>R </li></ul><ul><li>Alternative Treatment Design </li></ul><ul><li>R O X1 O </li></ul><ul><li>R O X2 O </li></ul><ul><li>R O O </li></ul><ul><li>Dismantling Studies </li></ul><ul><li>R O X1,2 O </li></ul><ul><li>R O X1 O </li></ul><ul><li>R O X2 O </li></ul><ul><li>R O O </li></ul>
  17. 17. Third Question <ul><li>A researcher at Spokane Mental Health </li></ul><ul><ul><li>Has parents of autistic children complete the Parental Stress Index (PSI) </li></ul></ul><ul><ul><li>Randomly assigns the parents into a behavior management course </li></ul></ul><ul><ul><li>Collects posttest PSI from all parents after the course is completed </li></ul></ul><ul><ul><li>Compares the groups on parental stress. </li></ul></ul><ul><li>What Design was used to answer your sites research question? </li></ul>

×