Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Comparing Research Designs


Published on

This is the handout version of a lecture I give to medical residents and fellows on the basics of clinical research designs and the inherent issues that go along with each one. I give this lecture as part of a multi-module lecture series on research design and statistical analysis.

Published in: Health & Medicine, Education
  • Would you please share this for knowledge enhancement of self and, to educate some students?
    Are you sure you want to  Yes  No
    Your message goes here

Comparing Research Designs

  1. 1. Comparing Research Designs Patrick BarlowStatistical and Research Design Consultant, Graduate School of Medicine, UTK PhD Student in Evaluation, Statistics, and Measurement, UTK
  2. 2. On the Agenda Common problems in research design  Confounding  Bias  Reliability  Validity Observational Research Designs  Cross-Sectional study  Case-control study  Cohort study Experimental Research Designs  Randomized Control Trial
  3. 3. Common Problems in Research Design Confounding Bias Reliability Validity
  4. 4. Confounding A confounder is a variable that is causally associated with the outcome (DV) and may or may not be causally associated with the exposure (IV) Causes spurious conclusions & inferences to be made about a set of variables Reduced through  Randomization  Matching  Statistically controlling (covariates)
  5. 5. Confounding ObesityPMH Use ? Colorectal Adenomas
  6. 6. Bias in Research The result of systematic error in the design or conduct of a study Can artificially “trend” results  Toward the Null hypothesis  Toward the Alternative hypothesis A major problem to consider when planning any study
  7. 7. Common Biases Selection bias: one relevant group in the population (e.g. cases positive for predictor variable) has a higher probability of being included in the sample Ascertainment: bias in asking questions or offering tests of one group over another Information: bias from erroneously classifying people in exposure/outcome categories Adjudication: bias in determining if the treatment was helpful due to partial or inadequate blinding Recall/Response: bias associated with inaccurate recall of exposure or representation of true exposure (self- report) Experimenter/Interviewer bias: Differential treatment of participants in treatment and control groups Publication: the tendency to publish only “positive” or “significant” findings.
  8. 8. Reliability• Refers to the consistency of an instrument/measurement.• Thought of as an individual’s “true score” on the phenomenon you aim to measure minus “measurement error”• Two common types of reliability o Internal consistency: Cronbach’s alpha, KR20 o Inter-Rater: Kappa statistic• Necessary but not sufficient in determining validity.
  9. 9. Reliability
  10. 10. Validity  Refers to the accuracy of an instrument/measurement  In other words, “the degree to which you’re measuring what you claim to measure”  Two broad types of validity  Internal validity  External validity
  11. 11. Internal Validity Concerns the accuracy of measurement within the study Can be threatened by  Biases  Confounding  History: large scale events that change participants’ attitudes or behavior (e.g., recession)  Maturation: participants change over time, e.g., growth, fatigue etc.  Repeated Testing: participants get wise to the study and remember the test questions  Compensatory rivalry/resentful demoralization: Control participants work extra hard to prove themselves or withdraw because not getting treatment  Diffusion: treatment effects spread from treatment group to control group
  12. 12. External Validity The ability to generalize the findings of your study to the relevant population. Threatened by  Bias  Confounding  Non-experimental design (i.e. case-control vs. RCT)  Lack of randomization External validity is the strongest when a true experimental design is used.
  13. 13. Comparing Research Designs Cross-Sectional Studies Case Control Studies Cohort Studies Randomized Control Trial (RCT)
  14. 14. Pyramid of Clinical Evidence EvidenceSystematic Reviews Summaries & Meta-analyses RCT Level 1 Evidence Cohort Studies Level 2 Evidence Cross-Sectional Case ControlStudies: Level 2.3 Studies Level 3 Evidence Case Series Case Reports Ideas, Editorials, Opinions Animal research In vitro (‘test tube’) research
  15. 15. Cross-Sectional Studies “Snapshot” of a population. People are studied at a “point” in time, without follow-up. Strength of evidence… What are some research questions that can be answered with cross-sectional designs?
  16. 16. Advantages and Disadvantages of Cross-Sectional Studies Advantages Disadvantages Fast and inexpensive  Can’t determine No loss to follow-up causal relationship Springboard to  Impractical for rare expand/inform diseases research question  Risk for nonresponse Can target a larger sample size
  17. 17. Case-Control Studies Always retrospective  Prevalence vs. Incidence A sample with the disease from a population is selected (cases). A sample without the disease from a population is selected (controls). Groups are compared using possible predictors of the disease state.
  18. 18. Advantages and Disadvantages of Case-Control Studies Advantages Disadvantages  Cannot estimate High information yield incidence of disease with few participants  Limited outcomes can Useful for rare be studied outcomes  Highly susceptible to biases
  19. 19. Strategies for Sampling Controls Population versus hospital/clinic-based controls Matching  Individual level  Group level Using 2 or more control groups
  20. 20. For Discussion“How much does a family history ofalcoholism increase the risk of being analcoholic?” The PI plans a case-controlstudy to answer this question.  How should she pick the cases?  How should she pick the controls?  What are some potential sources of bias in the sampling of cases and controls?
  21. 21. Cohort Studies A “cohort” is a group of individuals who are followed or traced over a period of time. A cohort study analyzes an exposure/disease relationship within the entire cohort. Groups selected based on exposure to a risk factor.
  22. 22. Cohort Design
  23. 23. Are U.S. Athletes more likely to win aGroup of gold medal than Chinese athletes atInterest the 2012 Olympics?(U.S.) Follow over the games Compare OutcomesComparisonGroup(China)
  24. 24. Prospective versus Retrospective Cohort Studies Exposure Outcome Assessed at the Followed intoProspective beginning of the the future for study (present) outcome Assessed at some Outcome hasRetrospective point in the past already occurred
  25. 25. Advantages and Disadvantages of Cohort Studies Advantages Disadvantages Establish population-based  Lengthy and costly incidence  May require very large samples Accurate relative risk  Not suitable for rare/long-latency Temporal relationship inferred diseases Time-to-event analysis possible  Unexpected environmental changes Used when randomization not  Nonresponse, migration and loss-to- possible follow-up Reduces biases (selection,  Sampling, ascertainment and information) observer biases Can study multiple outcomes  Changes over time in staff/methods
  26. 26. Randomized Controlled Trial Considered the “gold standard” by much of the research community (level 1 evidence) Blind vs. double blind Randomization Cause & effect
  27. 27. Designs of RCTs• Parallel Group Trial: Patients in the same randomized group throughout the study• Cross-over Trial: Patient randomly assigned to one group then crossed over to other group at some point. Patient serves as own control – greatly reduces sources of bias and confounding• Factorial Trial: Two or more interventions in a single experiment.
  28. 28. Disadvantages of RCT Designs Extremely time and resource demanding Unethical in many situations Poor external validity if the RCT is too highly controlled Difficult to study rare events Therapeutic misconception
  29. 29. Material Learned  Common problems in research design  Confounding  Bias  Reliability  Validity  Observational Research Designs  Cross-Sectional Studies  Case Control Studies  Cohort Studies  Experimental Research Designs  RCT DesignQuestions?
  30. 30. In Pairs… Work together to brainstorm an example of how your topic could be addressed using 1) a Cross- Sectional design, 2) a case-control design, 3) a prospective or retrospective cohort design, and an RCT. Be prepared to share your responses