Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Research Methods: Design and Analysis

1,203 views

Published on

Research Methods: Design and Analysis. Covering the research cycle, research questions, operationalization of variables, literature review, research designs, sampling method, instrumentation, data collection, validity, reliability, data analysis plan, and sample size

Published in: Education, Technology
  • Be the first to like this

Research Methods: Design and Analysis

  1. 1. Research Methods: Design and Analysis by Dr. James Lani
  2. 2. Today’s Webinar • • • • • • • • • • • • • • Why Do Research? The Research Cycle Research Questions and (Testable) Hypotheses Operationalization of Variables Literature Review Resources Research Designs Sampling Method and Sample Assignment Instrumentation Method of Data Collection (Procedures) Validity • Internal Validity • External Validity Reliability Data Analysis Plan Sample Size Q&A To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  3. 3. Why Do Research? OK, to get out of school….and also to examine questions we care about. We want to test a theory, evaluate a program or intervention, describe a phenomenon. Does Violence on TV cause children to be more or less aggressive? Do opposites attract or birds of a feather flock together? Does treatment X predict academic achievement? Is there a difference on depression scores by group (exercise vs. not)? What is True??? Goal of research: describe, explain, and predict. To establish knowledge, you need methods and statistics. To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  4. 4. The Research Cycle Literature Review Method Discussion Hypotheses and Operational Variables Results Method Analyze data and accept or reject Ho’s Design Study and Collect Data To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  5. 5. Research Questions and (Testable) Hypotheses • Hypotheses: State relationships among variables (constructs). Typically in terms of Differences, Relationships, or Prediction. • Keep research questions and hypotheses aligned (in number and language). • Questions must be testable. • Keep it tentative: no proof—just support or nonsupport To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  6. 6. Operationalization of Variables (or how do you measure what you’re studying?) Constructs—objects that are not directly observable • Constructs are measured with variables • Should be quantifiable • (e.g., depression can be measured with a Beck depression scale or by the observation of the number of times someone laughs) Confounding Variables • Provide alternative explanation of response in the DV Levels of Measurement: Nominal, Ordinal, Interval, and Ratio To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  7. 7. Literature Review Resources See: UW—Madison—the writing center UC Santa Cruz—UNW library—write a literature review Washington and Lee University—literature review Statistics Solutions’ Free membership—sample dissertation To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  8. 8. Research Designs To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  9. 9. Research Designs Its about how you structure your inquiry To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  10. 10. Research Designs Experimental vs. Non-experimental Designs Non-Experimental Designs Correlational studies, observational studies, quasi-experimental studies, and surveys Experimental (or True) Designs Manipulate the IV (e.g., Control vs. Experimental groups) Random assignment Confounds are controlled or eliminated To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  11. 11. Research Designs Experimental Designs One shot case studies X O One group pretest posttest O1 X O2 Statistic group X O1 O2 R=random assignment, O=observation, X=treatment group To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  12. 12. Research Designs True Experimental Designs Pretest-posttest control group design R O1 X O2 R O3 -- O4 R=random assignment, O=observation, X=treatment group Group Time3 Mean X Mean Control Mean Mean No diff Assignment Time 2 Treatment Random Time 1 Sig diff 10 5 Control 0 Treatment Pretest Posttest To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  13. 13. Research Designs For great research design references, see:      Cook and Campbell ‘79 Campbell and Stanley ‘63 Creswell ‘05 Leedy and Ormrod ‘10 SocialResearchMethods.net To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  14. 14. A word about Sampling Method and Sample Assignment Random sample: each individual in the population has an equal chance to become a participant of the sample. Random assignment: each participant of the sample has an equal chance of being assigned to a group (treatment or control). To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  15. 15. Instrumentation These are the tools that measure the outcomes—typically variables in the tools are categorized into independent variables (IV’s) and dependent variables (DV’s). IV’s (or treatment variables) are typically manipulated by researcher. DV’s (also called outcome variables or criterion variables) are responses to IV, caused by IV, or predicted by IV. The variables should be reliable and valid. Don’t make your own. See Google scholar, review articles, Statistics Solutions directory. To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  16. 16. Instrumentation        Questionnaires Interviews Observations Rating scales Achievement tests Personality inventories Historical reports or data To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  17. 17. Method of Data Collection (Procedures) After describing the sample and the instruments, tell the reader how you are going to systematically and objectively administer these instruments to these participants. To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  18. 18. Validity You measure what you say you are measuring. (Validity is more important than reliability) To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  19. 19. Internal Validity Eliminate confounds or competing explanations of differences on DV (E.g., Differences on Self-efficacy by gender…but the woman were 35 y.o. and men were 75 y.o.) Threats to internal validity • Selection bias—you think they represent the population (Safeguard tip: randomly sample and randomly assign) • Pre-post studies without a control group o History/maturation—something else occurred between the assessments • Drop-out/attrition • Reliability of measure • Low power • Order effect • Multiple tests of significance • John Henry Effect To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  20. 20. External Validity Def: The ability to generalize outcomes to the population Threats • Population validity—ability to generalize to population. (E.g., Kohlberg moral development— males value individual rights, where Gilligan found that females valued relationships). • Ecological validity—ability to generalize from lab to everyday environments. To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  21. 21. Reliability  Inter-rater reliability (kappa coefficient)  Test-retest reliability  Internal consistency—Cronbach’s alpha To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  22. 22. Data Analysis Plan (what to include) Reliability of scales: Cronbach’s alpha Descriptive statistics: Means, standard deviations, frequency and percentages. A data analysis plan for EACH research hypothesis. • Inferential: ANOVA, MANOVA, regression, timeseries, SEM, Chi-square, correlation • Non-inferential: descriptives (e.g., M, SD, %) Assumptions of analyses and justification for analyses To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  23. 23. Sample Size N=? 1. Effect Size—importance of differences 2. Power—likelihood of finding differences (typically .80) 3. Alpha (typically .05) To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  24. 24. 1-1 Consulting Info@StatisticsSolutions.com 877-437-8622 To schedule a consultation call 877-437-8622 or email Info@StatisticsSolutions.com
  25. 25. We’re interested… Webinar Feedback and What You’d Like Please provide feedback on the webinar and tell us what else you’d like to learn about in upcoming webinars. Send comments to: James@StatisticsSolutions.com
  26. 26. Thank You for your Participation and Attention! Join us for our next webinar on Wednesday, November 20th at 8:30pm ET Results Chapter: Conducting, Interpreting, and Writing

×