1. Educational Research 102:Selecting the Best Study Design for your Research Question Francis S. Nuthalapaty, MD 2010 APGO Faculty Development Seminar

2. Disclosures No relevant financial disclosures to declare

3. Learning Objectives Describe types of research and study designs Understand the characteristics of a good research question Understand how to convert the research question into a hypothesis Understand how to select the most appropriate study design to test the hypothesis

4. What do we already know? TRUE FALSE 1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL and Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher 27, 2, pp 10-28. 2. Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF Jr, Saint S. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. 2004 Dec 7;141(11):851-7.

5. Write an educational research question in which you have interest

6. Types of Research Empirical vs. Non-empirical Basic vs. Applied

7. Empirical vs. Non-Empirical Research Empirical Involves collection of data first hand Non-Empirical No first hand data collection

8. Example: Empirical or Non-empirical? The Effect of House Staff Working Hours on the Quality of Obstetric and Gynecologic Care Bailit, J et al, Obstet Gynecol 2009 OBJECTIVE: To measure the effect of house staff working hours reforms on the quality of obstetric and gynecologic care.

9. Example: Empirical or Non-Empirical? Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Issenberg SB et al, Med Teach 2005 OBJECTIVE: Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.

10. Types of Research Empirical vs. Non-empirical Basic vs. Applied

11. Basic vs. Applied Research Basic Results apply to a great many people and situations Results are related to general theory or to a general field of knowledge Results need not have immediate or even clear implications Applied Results are applicable only to a specific group of people in a particular situation. Results are not necessarily related to a broader field of knowledge Results must have immediate and clear implications for practice

12. Example: Basic or Applied? Assessing Vaginal Surgical Skills Using Video Motion Analysis Diwadkar G et al, Obstet Gynecol 2009 OBJECTIVE: To demonstrate the feasibility of using video motion analysis to quantitate a key step of vaginal hysterectomy and define measurable differences between novice and experienced surgical trainees during vaginal hysterectomy.

13. Example: Basic or Applied? Effects of a Depression Education Program on Residents’ Knowledge, Attitudes, and Clinical Skills LearmanL et al, Obstet Gynecol 2003 OBJECTIVE: To determine whether an interactive educational program would improve obstetrics and gynecology Residents' knowledge, attitudes, confidence, and skills in caring for depressed patients.

14. Research Methods What is the difference between Qualitative Research and Quantitative Research?

15. Qualitative vs. Quantitative Research Qualitative Descriptive and exploratory focus Used to gain insight into attitudes, behaviors, values Analysis of unstructured information Narrative reporting Quantitative Hypothesis driven Used to identify association and/or causation Analysis of discrete variables Statistical reporting

16. Quantitative Research Experimental & Quasi-experimental Non-Experimental Causal-comparative Correlational

17. Quantitative Research Experimental & Quasi-experimental Assess effect of an independent variable on dependent variables Comparison of 2 or more groups Control over ‘treatment’ & measurement Randomization Control group

18. Quantitative Research Designs Experimental Treatment Measurement Study Population Randomize Control Measurement Post-test only Control Group

19. Quantitative Research Designs Experimental Treatment Measurement Measurement Study Population Randomize Control Measurement Measurement Pre-test/Post-test Control Group

20. Quantitative Research Designs Experimental Treatment Measurement Measurement Control Measurement Measurement Study Population Treatment Measurement Randomize Control Measurement Solomon Four Group

21. Quantitative Research Designs Quasi-Experimental Study Population Treatment Measurement One-shot Case Study

22. Quantitative Research Designs Quasi-Experimental Study Population Treatment Measurement Measurement One-group Pre-test/Post-test

23. Quantitative Research Designs Quasi-Experimental Study Group 1 Treatment 1 Measurement Study Group 2 Treatment 2 Measurement Static Group

24. Quantitative Research Designs Quasi-Experimental Study Group 1 Measurement Treatment 1 Measurement Matching Study Group 2 Measurement Treatment 2 Measurement Pre-test/Post-test Matched Subjects

25. Quantitative Research Non-Experimental Causal-comparative Correlational

26. Causal-Comparative Baseline differences are pre-existing Non-permutable: Ethnicity, Gender Permutable: Teaching style Determine the cause or consequences of differences Associations can be identified Causality cannot be determined

27. Quantitative Research Designs Causal-Comparative Exposures Outcome Control Outcome Exploration of Effects

28. Quantitative Research Designs Causal-Comparative Exposures Outcome Exposures Control Exploration of Causes

29. Quantitative Research Designs Causal-Comparative Exposure Outcome Group 1 Outcome Group 2 Exploration of Consequences

30. Correlational Research Single group of subjects Describe degree to which 2 or more quantitative variables are related Help explain important human behaviors Predict likely outcomes Identifies associations Causality cannot be determined

31. Correlational Research

32. Correlational Research Correlation Coefficient Calculated by regression < .35 = only a slight relationship .40 - .60 = possible theoretical value .50 = minimum for crude predictions >.65 = reasonably accurate predictions >.85 = close relationship Positive correlation = direct association Negative correlation = inverse association

33. Threats to Internal Validity Subject characteristics Loss of subjects Maturation Repeated measures Statistical regression Investigator bias

34. External Validity Can the findings from the study be generalized to larger populations? Subject characteristics Environment (lab vs. natural setting) Did the subjects act differently b/c they were enrolled in a study (Hawthorne effect)?

35. What are the steps in the research process?

36. The Research Process Define Research Question Hole in the literature No hole in the literature Conduct Literature Review Refine Research Question Obtain IRB Approval Design Study Disseminate Results Collect Analyze Data

37. FINER Research Questions F – Feasible I – Interesting N – Novel E – Ethical R - Relevant

38. FINER Research Questions Feasible: Can be investigated with available: Time Money Space Other resources

39. FINER Research Questions Interesting: Interesting to you Interesting to others in the field Novel: Will answers to the question advance knowledge in the field?

40. FINER Research Questions Ethical: Will any physical or psychological harm come to anyone as a result of the research? Protect participants from harm Ensure confidentiality of data Avoid knowing deception of participants Relevant: How might answers to this question improve educational practice?

41. What is a research hypothesis and how is it different from a research question?

42. Research Hypothesis A prediction regarding the possible outcomes of the study Wallen & Fraenkel. Educational Research: A Guide to the Process. 2nd Ed.

43. Research Hypothesis A statement which: Summarizes the elements of the study The sample The design The predictor and outcome variables Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

44. Research Hypothesis Based upon a FINERresearch question Simple – one difference or relationship Specific – clearly stated, defined Stated in advance – before data collection Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

45. Research Hypothesis May be stated: As the alternative hypothesis Non-directional The difference is unknown Directional hypothesis Existing studies/data suggest direction of relationships As the null hypothesis Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

46. Research Hypothesis Ha: There is a difference between groups or variables Female medical students have a different IQ than male medical students μf ≠ μmorμf > μm H0: There is no difference between groups or variables Female medical students have a different IQ than male medical students μf = μm Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

47. Example: Hypothesis Improving Resident Competency in the Management of Shoulder Dystocia With Simulation Training Deering S, et al, Obstet Gynecol 2004 METHODS: Residents from 2 training programs were randomized by year-group to a training session on shoulder dystocia (SD) management that used an obstetric birthing simulator or to a control group with no specific training. Both groups were subsequently tested on a standardized SD scenario, and a physician grader rated the resident's performance with a standardized evaluation sheet.

48. Example: Hypothesis What is the research question? What is the null hypothesis? What is the research study design? Basic or Applied? Qualitative or Quantitative? Experimental or Non-Experimental

49. Quantitative Research Designs Experimental Treatment Measurement Study Population Randomize Control Measurement Post-test only Control Group

50. Small Group Activity Share your research questions Select one research question to convert to a hypothesis Select the most appropriate study design

51. Give Your Research Impact Investigate important questions Connect your study to prior studies and help build the body of work Uses appropriate research design Consult with a biostatistician in the design phase Dissemenate your results

53. Session content, including narrated MS Powerpoint slides available at: http://www.obgynknowledgebank.net