Experimental, Quasi experimental, Single-Case, and Internet-based Researches in education


Published on

Published in: Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Experimental, Quasi experimental, Single-Case, and Internet-based Researches in education

  1. 1. Experimental, Quasiexperimental, Single-Case Research and Internet based experiments And Article Critique Hatice ÇİLSALAR-Yelda SARIKAYA-ERDEM
  2. 2. Experimental Research Definition: Testing an idea to determine whether it influences an outcome or dependent variable. Key Characteristics:  Random Assignment: Process of assigning individuals at random to groups or to different groups  Control over Extraneous Variables: Controlling influences of selection of participants, the procedures, the statistics, the design likely to affect the outcome. Pretest-posttest, covariates, Matching Participants, Homogenous samples, Blocking variables  Manipulation of Treatment Conditions: Steps-Identify a treatment variable and its levels or conditions, manipulate treatment conditions  Outcome measures: Dependent variable that is the presumed effect of the treatment variable.  Group comparisons: Obtaining scores for individuals or groups on the dependent variable and comparing the means and variance between the groups.  Threats to validity: History(time passes), Maturation, Selection, Mortality, Interaction, Testing, et. Cresswel, (2014); Frankel, Wallen & Hyun, (2012)
  3. 3. Experimental Research A ‘true’ experiment includes several key features: one or more control groups one or more experimental groups random allocation to control and experimental groups pretest of the groups to ensure parity post-test of the groups to see the effects on the dependent variable one or more interventions to the experimental group(s) isolation, control and manipulation of independent variables Cohen, Mannion, & Morrison (2007)
  4. 4. Experimental Research Cresswel, (2014); page:334
  5. 5. Experimental Research How to Design an Experimental Research Define your research objectives Formulate hypotheses: H0 and H1 Set up your research design Select instruments Select appropriate levels at which to test your hypotheses Assign persons to groups randomly Carry out the experiment meticulously Analyze the data Muijs(2004); page:334
  6. 6. Experimental Research True Experimental Designs: 7 Pretest-Posttest Controlled Experimental Group Design Experiment group R(Random O1(Observation) X (treatment) Assignment) Control group R O3 O2 O4 Two control Groups and One Experimental Group Pretest-Posttest Design Experiment R O1 Control R O3 Control R X O2 O4 X O5 The Posttest Control-Experiment Group Design Experiment R Control R X O1 O2 Cohen, Mannion, & Morrison (2007)
  7. 7. Experimental Research The Posttest Two Experimental Group Designs Experiment R X1 O1 Experiment R X2 O2 The Pretest-Posttest Two Experiment Groups Design Experiment R O1 X1 O2 Experiment R O3 X2 O4 Matched Pairs Design Factorial Design Low Receive Health Lecture Smoking Number Medium Receive Health Lecture Smoking Number High Receive Health Lecture Smoking Number Low Receive Standard Lecture Smoking Number Medium Receive Standard Lecture Smoking Number High Receive Standard Lecture Smoking Number Cohen, Mannion, & Morrison (2007)
  8. 8. Experimental Research Parametric Design Poor Readers Token Number of Correct Word Average Readers Token Number of Correct Word Good Readers Token Number of Correct Word Outstanding Readers Token Number of Correct Word Control Number of Correct Word Repeated Measures Design G1 O X1 O X2 O X3 O G2 O X2 O X3 O X1 O G3 O X3 O X1 O X2 O G4 O X2 O X1 O X3 O G5 O X3 O X2 O X1 O Cohen, Mannion, & Morrison (2007)
  9. 9. Experimental Research Poor Experimental Designs: One-shot Case Study X O One-Group Pretest-Posttest Design O The Static-Group(Non-Equivalent) X X O O Comparison Design: The Static-Group(Non-Equivalent) Pretest-Posttest Design: O O O X O O Frankel, Wallen, & Hyun, 2012
  10. 10. Experimental Research True Experimental Designs: The Randomized Posttest Only Control Group Design Treatment R Control X O R O The Randomized Pretest-Posttest Only Control Group Design Treatment R O Control R X O O O The Randomized Solomon Four Group Design Treatment R O X O Control R O C O Treatment R X O Control R C O Random Assignment with Matching Frankel, Wallen, & Hyun, 2012
  11. 11. Experimental Research Single Group Designs The One-shot Case Study One group pretest-posttest design Time series designs Control Group Design with Random Assignment Pretest-posttest control group design Posttest only control group design One-variable multiple condition design Gall, Gall, &Borg, 2003
  12. 12. Experimental Research Between Group Designs True experimental design: (Randomized)Pretest-Posttest design or Posttest only design Quasi experimental design: (Un-randomized)Pretest-Posttest design or Posttest only design Factorial design Within Group/Individual Designs Repeated measures design: Interrupted(One experiment) or Equivalent (More than one experiment) Single subject designs: Multiple baseline design or Alternating treatments Creswell (2014)
  13. 13. Experimental Research Strengths: Causality: The best type for testing hypotheses about cause-and-effect relationships Manipulation of independent variable Help to see whether the treatment made difference. Go beyond description and prediction, beyond the identification of relationship-what causes them. Frankel, Wallen & Hyun, (2012)
  14. 14. Experimental Research Limitations: Difficult to Control some variables Address all threats Ethical issues: Control group may be disadvantaged by not receiving treatment or vice versa.
  15. 15. Quasi-experimental “quasi” means, in essence, “sort of.” = quasiexperiment is a “sort of” experiment. Definition: A quasi-experiment is a study that includes a manipulated independent variable but lacks important controls (e.g., random assignment), or a study that lacks a manipulated independent variable but includes important controls. Includes nonrandom assignment-matching. More threat to internal validity: maturation selection, mortality, interaction of selection, history, testing, instrumentation, regression- Cresswell (2014)
  16. 16. Quasi-Experimental Research How to Design an Experimental Research Define your research objectives Formulate hypotheses: H0 and H1 Set up your research design Select instruments Select appropriate levels at which to test your hypotheses Assign persons to groups randomly (only experimental design) Carry out the experiment meticulously Analyze the data Muijs(2004); page:334
  17. 17. Quasi-experimental Types: A Pre-experimental Design: The one group pretest-posttest O1 X O2 A Pre-experimental Design: The one group posttest only design X O1 A Pre-experimental Design: The posttests only non-equivalent groups design A Quasi-experimental design: The pretest-posttest nonequivalent groups design Experimental O1 X O2 Comparison O3 O4 The One Group Time Series
  18. 18. Quasi-experimental Cresswell (2014)
  19. 19. Single-Case Research- Definition Key Features: Single - one subject Standard conditions Repeated measurement Effectiveness or productivity Three components: (a) repeated measurement, (b) baseline phase, and (c) treatment phase. alternative to group designs. Group designs compare the performance of one sample of individuals (e.g., people who don’t smoke, or rabbits who don’t have smoke blown into their cages) with another (e.g., people who do smoke, or rabbits who do have smoke blown into their cages). Single-subject designs compare the performance of an individual before and after a specified intervention. Alberto& Troutman, 1995;Best& Khan, 1998,Tekin (2002),
  20. 20. A-B Design Regardless of the research design, the line graphs used to illustrate the data contain a set of common elements. Dependent measure Condition identifications Baseline 8 Praise 7 Frequency of disruptions Independent variable Condition change line 6 5 4 3 Ordinate Data points Data path 2 Abscissa 1 0 0 1 2 3 4 Unit of time 5 6 7 8 Day 9 10 11 12 13 14 15 16 Measure of time
  21. 21. Single-Case Research- Types A-B-A-B Designs: Reversibility-last experimental control or no functional relationships Number of fulfilled assignments and without token(A) and treatment with tokens(B). (Choen, Mannion, & Morrison, 2007; Kennedy, 2005)
  22. 22. Single-Case Research- Types B-A-B Designs: an intervention already placed Sometimes an individual’s behavior is so severe that the researcher cannot wait to establish a baseline Or an intervention already placed so researcher must begin with an intervention. In this case, a B-A-B design is used. The intervention is followed by a baseline followed by the intervention.
  23. 23. Single-Case Research- Types B A B Praise 8 Baseline Praise Frequency of disruptions 7 6 5 4 3 2 1 0 0 1 2 3 4 5 6 7 8 Day 9 10 11 12 13 14 15 16
  24. 24. Single-Case Research- Types A-B-C Designs: additional opportunity to analyze how various interventions influence behaviors 18 16 Earns Candy Number Correct 14 12 10 Earns Money 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Sessions Kennedy, (2005) 17
  25. 25. Single-Case Research A-B-C Designs: Instructor feedback Peer feedback
  26. 26. Single-Case Research Strengths Researcher can establish a cause-and-effect relationship between treatment and behavior using only a single participant See the effect of a treatment on a single participant Flexibility – development of the design depends on participant’s responses By using comparative designs, compare and contrast the results of the studies easily
  27. 27. Single-Case Research Limitations Problem with generalizations since designs use only one participant Multiple observations can affect participant’s responses Absence of statistical controls and reliance on visual inspection of the data
  28. 28. Internet based experiments Three data collection method through Internet; Nonreactive data collection Online Surveys Web based experiments (Reips,2002)
  29. 29. Internet based experiments Why? Speed, Low cost, Experimenting around a clock, A high degree of automation of the experiment, a wider sample. Large number of participants High statistical power Protection of anonymity Huge representativeness There is little evidence in the literature that Internetbased surveys achieve higher response rates, as a general rule, than conventional surveys Reips (2002)
  30. 30. Internet based experiments Form of emails to emails-plus-attachments of the questionnaire itself, to emails directing potential respondents to a web site, or simply to web sites. Although email surveys tend to attract greater response than web-based surveys, web-based surveys have the potential to reach greater numbers of participants Page layout options should be simple not advanced Avoid open-ended questions not to distrupt participants attention Confirming of each item can be difficult for those who have less developed computer skills. Keep the introduction to the questionnaire short (no more than one screen), informative (e.g. of how to move on) and avoiding giving a long list of instructions. Keep the response categories close to the question Cohen,
  31. 31. Internet based experiments Advantages: Ease access to a large number of demographically and culturally diverse participants Specific participant population Better generalizability of findings to population, more settings or situations Avoidance of time constrains, organizational problems: scheduling difficulties, as thousands of participants may participate simultaneously Reduction of experimenter effects, demand characteristics Cost saving of personnel hours, equipment, administration Greater openness of the research process Access to the number of nonparticipants Ease access for participants Public control of ethical issues Highly voluntary participation High participation: High statistical power Detectability of motivational Reips (2002)
  32. 32. Internet based experiments Disadvantages: Possible multiple submission: warning about multiple submission, blocking using same IP address, or handing out passwords-one time password, participant pool or online panel, control by collecting personal identification, controlling internal consistency Self-selection: can be controlled by using the multiple site entry technique. Dropout: Promising immediate feedback, giving financial incentives, by personalization Misunderstood instructions: Pretest of materials and providing the participants with the opportunity for giving feedback The comparative basis is relatively low. External validity is limited by their dependence on computer Reips, (2002)
  33. 33. Internet based experiments Dillman et al. (1999) three ways to overcome problem of differential expertise in computer usage: having the instructions for how to complete the item next to the item itself at the start of the questionnaire asking the respondents at the beginning about their level of computer expertise, and, if they are more expert, offer omitting instruction part and, if they are less experienced, directing them to instructions having a ‘floating window’ that accompanies each screen and which can be maximized for further instructions. Cohen
  34. 34. Internet based experiments Reips, (2002)
  35. 35. Internet based experiments 16 Standards: 5. Consider multiple site entries 1. Consider to use web-based 6. Run survey both online and software tool to create survey 2. Pretest the instrument for clarity of instructions availability on different platforms 3. Make a decision about offline for comparision 7. If dropout is to be avoided use the warm-up technique 8. Use dropout to determine whether there is motivational confounding advantages out-weigh the disadvantages 4. Check your web survey for configuration errors Reips, (2002)
  36. 36. Internet based experiments 16 Standards: 13. Perform consistency checks 9. Use high-hurdle technique, 14. Keep logs incentive information 10. Ask filter questions at the 15. Report and analyze drop out rates beginning of the experiment to encourage serious and complete 16. The experimental materials should be kept available on the responses. Internet, as they will often give 11. Check for obvious naming of a much better impression of files, conditions, passwords what was done than any verbal description could convey. 12. Use , if needed to avoid multiplication, participant tools or password techniques Reips, (2002)
  37. 37. References Cohen, L., Manion, L., & Morrison, K. (2013). Research methods in education. Routledge. Creswell, J. W. (2014). Educational research: Planning, conducting and evaluating, quantitative and qualitative. Pearson International Edition. Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education. McGraw-Hill International Edition. Gall, M. D., Gall J. P. & Borg, W. R. (2003). Educational research: An introduction. Pearson. Kennedy, C. H. (2005). Single-case designs for educational research. Financial Times/Prentice Hall. Reips U. D. (2002). Theory and techniques of web based experimenting. In B. Batinic, U.D. Reips, & M. Bosnjak (Eds.) Online Social Sciences. Seattle Hogrefe & Huber. Reips, U. D. (2002). Standards for Internet-based experimenting. Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie), 49(4), 243-256. Tekin, E. (2000). Karşılaştırmalı tek denekli araştırma modelleri. Özel Eğitim Dergisi, 2(4), 1-12.