Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Like this presentation? Why not share!

4,918 views

Published on

No Downloads

Total views

4,918

On SlideShare

0

From Embeds

0

Number of Embeds

3

Shares

0

Downloads

0

Comments

0

Likes

2

No embeds

No notes for slide

- 1. Fixed Designs Experimental &Quasi-Experimental Grant Heller, Ph.D. PYC 5040
- 2. Fixed Designs• Theory driven• Should always be piloted first• Manipulation check may be useful• Confirmatory (fixed design) vs. Exploratory approach• Reliability• Construct Validity – Face – Predictive criterion – Internal – External (generalizability)
- 3. Experimental Research• Issues o Random selection and assignment o Group equivalence• Control & Comparison groups o Control (no treatment, wait list, placebo) o Comparison (standard treatment)• Assessing the impact of the Intervention / Manipulation o Manipulation checks o Treatment fidelity
- 4. Variance• Total Variance = Systematic Variance + Unsystematic Variance – Systematic Variance = Treatment Variance + Confound Variance – Unsystematic Variance = Error Variance (within- groups variance)
- 5. Validity• Internal Validity o extent to which the changes in the study DV can be attributed to changes in the IV• External Validity o extent to which the results can be generalized
- 6. Threats to Internal Validity (Campbell & Stanley, 1963)• History• Testing• Instrumentation• Statistical Regression (to the mean)• Differential Mortality• Maturation• Selection• Selection X Maturation (interaction)• Experimenter Bias• Ambiguity about causal direction (A B or B A?)• Diffusion of treatments• Compensatory equalization of treatments• Compensatory rivalry
- 7. Threats to Internal Validity• Remember the acronym: MRS SMITH – Maturation – Regression to the mean – Selection of subjects – Selection by maturation interaction – Mortality – Instrumentation – Testing – History
- 8. Controlling for Threats to Internal Validity• Random assignment *** – Random assignment vs. Random selection• Matching – To ensure equivalency between groups• Blocking – To determine effects of extraneous variables• Holding extraneous variables constant – Reduces generalizability• Controlling for effects of extraneous variables (covariates) statistically – ANCOVA, MMR, partial correlation, etc.
- 9. Blocking example Therapy Wait List IQ > 110 IQ > 110Therapy Wait List Therapy Wait List IQ < 110 IQ < 110
- 10. Maximizing Internal Validity (Fraenkel & Wallen, 1993)1. Standardization of conditions – Minimize history & instrumentation2. Obtain as much info on participants as possible – Minimize mortality & selection3. Tighten up procedures of study – Minimize history & instrumentation4. Choose appropriate research design – Helps control most threats to internal validity
- 11. Threats to External Validity (generalizability) (LeCompte & Goetz, 1982)• Selection – Address through random selection• Setting• History• Construct effects
- 12. Threats to External Validity cont.• Selection X Treatment Interaction• History X Treatment Interaction• Testing X Treatment Interaction• Demand Characteristics• Hawthorne Effect• Order Effects (aka carryover effects & multiple treatment interference)
- 13. Ways to Increase External Validity• Random sampling/selection *** – Stratified random sampling – Cluster sampling• Naturalistic Research – Internal validity at expense of external validity – Analogue research• Single- and Double-Blind Research• Counterbalancing
- 14. Defense against threats to validity• for External Validity o Random selection of subjects• for Internal Validity o Random assignment to conditions
- 15. Internal vs. External Validity• Tradeoff between Internal & External Validity• How do we prioritize one over the other? – Most would argue in favor of internal validity• Mook (1983) In Defense of External Invalidity – “to what populations, settings, and so on, do we want the effect to be generalized?” (p. 379) – “we are not making observations, but testing them.” (p. 380) – Lab experiments allow us to test theory, find out what is possible, and break down phenomenon.
- 16. Specific Research Designs & Strategies• True Experimental Research – Random assignment to groups, receive different levels of manipulated variable• Quasi-Experimental Research – Random assignment is not possible (pre-existing groups)• Correlational Research – To be covered at a later date – Variables measured rather than manipulated• Developmental Research• Time-Series Design• Single-Subjects Designs• Qualitative Research – will covered
- 17. Design Notationhttp://www.socialresearchmethods.net/
- 18. Experimental Fixed Designs• Assignment of Ss to different conditions• Manipulation of at least 1 variable (IV)• Measurement of effects of manipulation on 1 or more variables (DV’s)• Control of all other variables• Experimental realism vs. Mundane realism• Demand characteristics – Deception: but at what cost?• Expectancy effects – Double blind procedures
- 19. 3 Essential Properties of a Well Designed Experiment (Leary, 2004)1. Manipulation of 1 or more Independent Variables (IVs)2. Random assignment to groups 1. Assure initial group equivalence3. Adequate control of extraneous variables
- 20. 3 Aspects of Experimental Design• 1.) the number of independent variables (IV’s)• 2.) the number of treatment conditions – Levels of IV’s• 3.) whether the same or different subjects are used in each treatment condition.
- 21. Types of Experiments• Between-Subjects Design• Within-Subjects Design (repeated measures)• Mixed-Design – Combines between & within subjects designs• Single-Subject Design
- 22. Three Pre-Experimental Designs• 1.) The one-shot case study X OVulnerable to: History, Maturation, Selection, Mortality, Selectio n X Treatment Avoid!
- 23. Three Pre-Experimental Designs• 2.) The one-group pretest-posttest design O1 X O2Vulnerable to: History, Maturation, Testing, Instrumentation, Regression (?), Selection X Maturation, Selection X Treatment Avoid!
- 24. Three Pre-Experimental Designs• 3.) The static group comparison __ __O1 X __ O2Vulnerable to: Selection, Mortality, Selection X Maturation, Maturation (?), Selection X Treatment Avoid!
- 25. Designs to Avoid• Post-test only design – Problem: impossible to determine change from pre- treatment (no baseline measure) – Suggestion: improve design or adopt case study methodology• Post-test only non-equivalent groups – Problem: no baseline measure, so any differences between groups cannot be attributed to treatment – Suggestions: incorporate a pre-test; employ random assignment when possible; consider case study• Pre-test post-test single group design – Problem: widely used, but vulnerable to history, maturation regression. – Suggestion: add 2nd pre-tested no-treatment control group
- 26. True Experimental Designs• Two group designs – Post-test-only randomized control trial (RCT) – Post-test-only two treatment comparison – Pre-test post-test RCT – Pre-test post-test two treatment comparison• Three (or more) group simple designs• Factorial designs• Parametric designs• Matched pairs designs• Repeated measures designs – Within-groups design
- 27. Three True Experimental Designs• 4.) The pretest-posttest control group design R O1 X O2 R O3 O4
- 28. Three True Experimental Designs• 5.) The Solomon four-group design R O1 X O2 R O3 O4 R X O5 R O6
- 29. Three True Experimental Designs• 6.) The posttest-only control group design R X O1 R O2
- 30. Within-Subjects Designs• Advantages • Disadvantages – Increased statistical – Order effects power • Address through • Fewer participants counterbalancing needed • Latin square design • Carryover effects may still exist 1st 2nd 3rd 4th Group 1 0 mg 100 mg 600 mg 300 mg Group 2 100 mg 300 mg 0 mg 600 mg Group 3 300 mg 600 mg 100 mg 0 mg Group 4 600 mg 0 mg 300 mg 100 mg
- 31. Posttest-Only One-Way Designs • Randomized groups design Random Initial IV DV assignment Sample manipulated measured to groups • Matched-subjects design Ss in each block Initial Matched randomly IV DV assigned to manipulated measuredSample into blocks groups • Repeated measures design Initial Receives 1 DV Receives DV another levelSample level of IV measured of the IV measured
- 32. Pretest-Posttest-Only One-Way Designs • Randomized groups design DV Random DV Initial IV manip- measured assignment measuredSample to groups ulated (pretest) (posttest) • Matched-subjects design Ss in blocks DV Match DV Initial randomly IV manip- measured into assigned to measuredSample ulated (pretest) blocks groups (posttest) • Repeated measures design DV Receive DV Receive DV Initial measured another measured measured one levelSample (pretest) of IV posttest level of IV posttest #1 #2
- 33. 2 X 2 Factorial Design Independent Variable A A1 A2 Also notated: R X11 OIndependent B1 Variable B R X12 O R X21 O B2 R X22 O
- 34. 3 X 2 Factorial Design Independent Variable A A1 A2 A3Independent B1 Variable B B2
- 35. 2 X 2 X 2 Factorial Design Independent Variable A A1 A2 Independent Variable B Independent Variable B B1 B2 B1 B2Independent C1 C1 Variable C C2 C2
- 36. 2 X 2 X 2 Factorial Design Same design, different notation A1 A2 B1 B2 B1 B2C1 C2 C1 C2 C1 C2 C1 C2
- 37. http://www.socialresearchmethods.net/
- 38. Factorial Design: the null outcome http://www.socialresearchmethods.net/
- 39. Factorial Designs: main effects 1 http://www.socialresearchmethods.net/
- 40. Factorial Designs: main effects 2 http://www.socialresearchmethods.net/
- 41. Factorial Designs: main effects 3 http://www.socialresearchmethods.net/
- 42. Factorial Designs: interaction effect 1 http://www.socialresearchmethods.net/
- 43. Factorial Designs: interaction effects 2 http://www.socialresearchmethods.net/
- 44. Experimental Designs: when to use• Matched designs – Matched variables correlate with DV; measurement of matched variable unlikely to influence treatment effect• Repeated measures designs – Order effects unlikely; IV’s lend to repeated measurement; would likely be exposed in real life; individual differences likely to mask treatment effects• Simple two group designs – Order effects likely; IV(s) don’t lend to repeated measurement; Ss may be sensitized by pretesting or matching; not likely to get all treatments in real life.• Before-after / pre-post design – Pre-testing unlikely to affect Tx effects; concerns whether random assignment has produced equivalent groups; individual differences may mask Tx effects• Factorial designs – Interested in > 1 IV & interaction effects a concern• Parametric designs – IV(s) have a range of values or levels of interest; wish to investigate form or nature of relationship between IV and DV
- 45. Field Experiments• Problems – Random assignment – Validity – Ethical issues – Control• Advantages – Generalizability – Validity – Participant availability
- 46. Quasi-experiments“A research design involving an experimentalapproach but where random assignment totreatment and comparison groups has notbeen used” (Campbell & Stanley, 1963).
- 47. Quasi-experiments• Experimental approach, but random assignment not used• Typically employ naturally occurring groups – Classrooms, clinics, organizations, geographic areas, etc.• Generally do not possess same degree of internal validity as true experiments
- 48. Common threats to internal validity of quasi-experimental designs• Pretest-posttest designs – History – Maturation – Regression (to the mean) – Pretest sensitization• Two or more nonequivalent groups – Selection bias – Local history
- 49. Quasi-Experimental Designs• 7.) The one-group pretest-posttest design O1 O2 O3 O4 X O5 O6 O7 O8 25 20 15 10 5 0 O1 O2 O3 O4 X O5 O6 O7 O8
- 50. Quasi-Experimental Designs• 8.) The equivalent time-samples design X1O X0O X1O X0OVulnerable to: Testing X Treatment, Reactive arrangements, Multiple treatment interference, Selection X Treatment (?)
- 51. Quasi-Experimental Designs• 10.) The nonequivalent control group design O __ __X __ __ __ __ O O O
- 52. Designs for Studying Development• Effects to consider – Age – Cohort – Time of measurement• Designs – Longitudinal – Cross-Sectional – Cross-Sequential
- 53. Longitudinal DesignCohort 2000 2010 2020 20301950 50 60 70 801960 40 50 60 701970 30 40 50 601980 20 30 40 50
- 54. Cross-Sectional DesignCohort 2000 2010 2020 20301950 50 60 70 801960 40 50 60 701970 30 40 50 601980 20 30 40 50
- 55. Cross-Sequential DesignCohort 2000 2010 2020 20301950 50 60 70 801960 40 50 60 701970 30 40 50 601980 20 30 40 50
- 56. Single Subjects Designs• AB design• Reversal or withdrawal design (ABA)• ABAB• Multiple Baseline design

No public clipboards found for this slide

×
### Save the most important slides with Clipping

Clipping is a handy way to collect and organize the most important slides from a presentation. You can keep your great finds in clipboards organized around topics.

Be the first to comment