Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Published in: Technology, Business
  • Be the first to comment


  1. 1. CHAPTER 13 EXPERIMENTAL RESEARCH By: David Cook, Anne D’Alonzo, Angela McCoy, Carla Oden, Kim Ross, Brandi Young,
  2. 2. Summary of Presentation Summary of Chapter 13: Experimental 1. Research Vocabulary Quiz/ Flash Cards 2. Article 20: Summary and Discussion 3. Class Experiment and Discussion 4.
  3. 3. Essential Characteristics One of most powerful methodologies  Establish cause–and–effect among variables  Not always Easy to conduct  Powers and Problems in conducting experiments  ONLY type of research that directly attempts to  influence a particular variable When applied properly – BEST type for testing  hypotheses about cause-and-effect relationships Researchers MANIPULATE the independent variable  Consist of 2 Basic Conditions:  2 or more conditions or methods are compared  The independent variable is directly Manipulated 
  4. 4. Comparison of 2 Groups of Subjects Experimental Group receives a treatment  Control Group receives no treatment  (medical/psych.) Comparison Group receives a different  treatment (ed.)
  5. 5. Manipulation of the Independent Variable Researcher actively manipulates the  independent variables and decides which group will get the form, when, where, and how. Ex. Transparencies vs. no transparencies in a statistics class.
  6. 6. Randomization (assignment & selection similar, not identical) Random Assignment: every individual has an equal chance to be  assigned to any of the experimental or control conditions. Each member is given an arbitrary number and a table of random numbers is used to select the members of the experimental and control groups. Takes place BEFORE the experiment begins  Process of assigning individuals to groups, not a result of the  distribution The researcher formed groups that at the beginning of the study are  equivalent; they differ only in chance. This is intended to eliminate the threats of EXTRANEOUS or additional variables – those which researchers are BOTH aware and unaware that might affect the outcome of the study. No guarantee of equivalent groups unless both groups are  sufficiently large (most researchers want no fewer than 40 subjects in each group). Random Selection: every member of a population has an equal  chance of being selected to be a member of the sample.
  7. 7. Control of Extraneous Variables Researchers must control any and all subject characteristics by  ensuring that the 2 groups are as equivalent as possible on all variables except the one(s) being studied (the independent variables). Randomization  Holding certain variables constant. Ex., eliminate one gender –  drawback is reduced generalizability. Building the variable into the design. Ex., include both genders and  analyze the effects. Matching. Ex., match by age and assign 1 member of each pair to  each of the comparison groups. Using subjects as their own controls. Ex., same students taught  algebra 1st by inquiry method than by lecture method. Using analysis of covariance. Equate groups statistically on the  basis of a pretest or other variable. Posttest scores are adjusted accordingly.
  8. 8. Group Designs in Experimental Research WEAK Experimental Designs:  Design can The One-Shot Case Study. Remedy: a comparison  take a could be made with another group. Seldom used. variety of Treatment … Observation  forms. 2. The One-Group Pretest-Posttest Design. A single  GOOD group is measured after being exposed as well as designs before. Remedy: a comparison group to be added. control many Pretest … Treatment … Posttest  of the threats to 3. The Static-Group Comparison Design or  Nonequivalent Control Group Design. Two already internal existing groups or static groups are used. NOT validity. randomly assigned. 4. The Static-Group Pretest-Posttest Design. Only  difference is that a pretest is given to both groups. Pretest score is subtracted from posttest score.
  9. 9. TRUE Experimental Designs: * Subjects are randomly assigned to treatment  Please Note this groups. This controls the subject characteristics distinction: threat to internal validity. Random Selection and 1. The Randomized Posttest-Only Control  Random Assignment differ Group Design. Two groups by random in purpose. assignment. One group gets experimental Random Selection is to treatment, other group does not. Both groups provide a representative are posttested on the dependent variable. sample. It may or Note: the threat of subject may not be accompanied by characteristics, maturation, and statistical the random assignment of regression are well controlled. None of subjects subjects to are measured twice, so testing is not a threat. groups. Random Assignment is BEST of all designs, provided there are at least intended to equate 40 subjects in each group. Mortality is a threat. groups, and often is not Hawthorne (attitudinal) threat is possible. accompanied by random selection.
  10. 10. TRUE Experimental Designs, cont. 2. The Randomized Pretest-Posttest Control  Group Design. This has a pretest. Two groups of subjects are used and both groups are measured twice. The use of a pretest raises the possibility of a pretest treatment interaction threat since it may “alert” members of the experimental group. The pretest does check whether the groups are really similar.
  11. 11. TRUE Experimental Designs, cont. 3. The Randomized Solomon Four-Group  Design. Designed to eliminate the possible effect of a pretest. It is random assignment of subjects to 4 groups, with 2 of the groups being pretested and 2 not. One pretested group and one unpretested group is exposed to the experimental treatment. All 4 groups are posttested. This is the BEST control of threats to internal validity. Requires a large sample and much work.
  12. 12. TRUE Experimental Designs, cont. 4. Random Assignment with Matching. Pairs  of individuals may be matched on certain variables. The members of each matched pair are then assigned to the experimental and control groups at random. Matching may be done mechanically or statistically – both require a score for each subject on each variable
  13. 13. Mechanical Matching: Pairing 2 persons whose scores on a particular  variable are similar. After the matching is completed for the entire sample, a check should be made with a frequency polygon to ensure that the 2 groups are equivalent on each matching variable. TWO Problems: 1) Difficult to match on more than 2 or 3 variables, making it necessary to have a very large sample. 2) Some subjects will need to be eliminated because there are no matches for them, samples are no longer random. 1 member of each matched pair is randomly assigned to experimental group, the other the control group.
  14. 14. Statistical Matching: Not perfect, however recommended over mechanical. Does  NOT lose subjects, nor does it limit the number of matching variables. The sample is divided randomly at the beginning and the statistical adjustments are made after the data have been collected. Each subject is given a predicted score on the dependent variable, based on the correlation between the dependent variable and the variable(s) on which the subjects are being matched. The difference between the predicted and actual scores for each individual is used to compare experimental and control groups. When pretest is matching variable, the difference between the predicted and actual score is called regressed gain score. This score is preferable to the straightforward gain scores (posttest minus pretest score for each individual) because it is more reliable. *Need random assignment on all variables with above 2. 
  15. 15. Quasi-Experimental Designs Do not include  the use of random assignment  Other techniques  control threats to internal validity 
  16. 16. The Matching-Only Design Random assignment is not used  Matches subject in experimental and control groups  No assurance matches are equivalent  This design offers an alternative to random  assignment when 10 or more groups are available for a method study Groups can be randomly assigned to different  treatments Individuals are matched with individuals receiving  other treatments Matching is never a substitute for random assignment  The correlation between matching variables and the  dependent variable should be substantial
  17. 17. Counterbalanced Designs In different orders all groups are exposed to all  treatments This design Counterbalanced designs represent another technique for  controls threats equating experimental and comparison groups to internal Many different treatments may be involved validity  A Three-Treatments Counterbalance Design involves  Internal three groups Validity: Group 1: treatment 1+posttesting observed  differences on treatment 2+ posttesting  the dependent treatment 3+posttesting variable are  directly related Group 2: treatment 2+posttesting  to the treatment 3+posttesting independent  variable and treatment 1+posttesesting  not due to Group 3: treatment 3+postest some other  unintended treatment 1+posttest  variable treatment 2+postesting  Group treatments are in random order  By comparing the average scores for all groups  researchers determine the effectiveness of various
  18. 18. Time-Series Designs Involves repeated measurements or  Involves observations over time  Both before and after treatments  Extensive amount of data collected  The Researcher has more confidence if the group  pretests and posttests cause improvement with multiple tests. Threats to internal validity endanger this design for  example history Effectiveness is by analyzing pattern  Time-series design is a strong design, although it is  vulnerable to history
  19. 19. Factorial Designs Extend the number of relationships that may  be examined in an experimental study Researchers may study interactions of an  independent variable Moderator variables may be either treatment  variables or subject characteristic Factorial Designs are an efficient way to study  several relationships
  20. 20. Control of Threats to Internal Validity: A Summary Time series design suffer from instrument  decay and data collector bias Unconscious bias on the part of data collectors  is not controlled by any of these designs Implementers or data collectors can  unintentionally distort the results of a study Regression is not likely to be a problem except  in the one-group pretest-posttest design
  21. 21. Evaluating the Likelihood of a Threat to Internal Validity Consider the likelihood of threats in an  experimental study A number of possible threats to internal validity  may exist Researchers must question possible threats to a  study by using the following procedures: Question factors related to the study affecting  dependent variables Question comparison groups differing the same  factors Evaluate the threats and plan to control them 
  22. 22. Threats to Internal Validity: Subject  Characteristics Mortality  Location  Instrumentation  History  Maturation  Attitude of Subject  Regression  Implementation 
  23. 23. Subject Characteristics to affect critical thinking ability  1. Initial critical thinking ability  2. Gender 
  24. 24. Mortality Location and data could affect scores  Group numbers should be verified  Have an effect unless controlled: moderate  high
  25. 25. Location Data collection and location differing for two  groups could affect posttreatment scores Threats may differ for groups in different  locations Likelihood of having an effect: moderate to  high
  26. 26. Instrumentation Instrument decay May affect any outcome  Could differ for groups  Unless controlled could have an effect  Data collector characteristics  Might affect critical thinking test scores  Use same data collectors to avoid decay  Likely to have an effect unless controlled  Data collector bias  Affects scores on critical thinking  Controlled by training implementers  Unless controlled likely to have an effect 
  27. 27. History Extraneous events  Affect both groups  Likely to have an effect unless controlled 
  28. 28. Maturation Affect outcome scores since critical thinking is  related to individual growth No threat if instructors teach over same time  period Likely to have an effect unless controlled 
  29. 29. Attitude of Subject Affect posttest scores  Perceived special attention could be a threat  Low moderate threat unless controlled 
  30. 30. Regression Will affect scores if subjects are selected on  the basis of extreme scores Treatment unlikely to affect groups differently  The likelihood of having an effect is low 
  31. 31. Implementation Instructor characteristics are likely to affect  posttreatment scores Different instructors teach different methods  control by monitoring instruction The likelihood of having an effect is high 
  32. 32. Identify threats to internal validity Control of Experimental Treatments Think of different  Improve internal validity of variables experimental study - Decide if  Researcher control well constructed these things would affect experiment things differently  Researcher controls the treatment based on the 5’ws + how evidence  Researcher controls testing - Threats need to be  Researchers seldom have total minimized control: problems must be faced……
  33. 33. An Example of Experimental Research Study Purpose/justification: explicit implications  Exemplify Definitions: clearly defined context typical  methodology Prior research: previous work connected  and permit constructive Hypotheses: stated, implied, appropriate  criticism Sample: target population indicated  Instrumentation: adequately described  Hold Procedures/internal validity: evident  students threats attention Data analysis: are statistics correct  Results: clearly presented  Be reported Discussion/interpretations: limitations  concisely recognized with regard to population
  34. 34. The Effects of a Computer Simulation Activity Versus a Hands-on Activity on Product Creativity in Technology Education Purpose/Justification: Study the effect of computer simulation  Definitions: The dependent variable, product creativity; is  confused by the discussion of creativity Prior Research: The brief summaries of two studies on  outcomes of computer graphics and word processors are well done, but we must assume they are the only ones Hypotheses: Clearly stated  Sample: A convenience sample was used  Instrumentation: Product creativity was measured, validity is not  adequately addressed Procedures/Internal Validity: Internal validity was  controlled, mortality did not occur Data Analysis/Results: Lack of random selection  Discussion/Interpretation: Results do not support “the use of  computer simulation to enhance product creativity.”
  35. 35. Article Summary Influence of Social Context on Reported  Attitudes of Nondisabled Students Towards Students with Disabilities  Article 20, page 149 Helpful research worksheet on iLearn 
  36. 36. Research Question and Hypothesis Question: Would the mere presence of a  student with a disability affect the responses of nondisabled students to a survey that assessed their attitudes toward people with disabilities? Hypothesis: Individuals would report a) a more  tolerant attitude toward persons with disabilities and b) less discomfort in a social setting when paired with a similar rather than dissimilar individual.
  37. 37. Operational Definitions and Variables Persons with disabilities: An individual who  uses a wheelchair to perform any daily living activities Independent variable:The presence of a  disabled person or non-disabled person. Dependent variable: Amount of comfort and  attitude towards disabled person
  38. 38. Sampling 30 Louisiana State University students   Ten people in the group have disabilities  Don’t know how sample was selected or how non-disabled participants were assigned to test groups Two confederates (one in wheelchair) 
  39. 39. Instruments Demographic questionnaire  Original Attitudes Toward Disabled People  Scale (ATDP-O) Likert-type scale to measure general ease  (GME)
  40. 40. Procedure Three experimental conditions  ND/ND: non-disabled (9 women/1 man; mean age = 22) and non-  disabled confederate D/ND: non-disabled (9 women/1 man; mean age = 22.5) and  disabled confederate D/D: disabled (4 women/6 men; mean age = 30.5) and disabled  confederate One by one, groups escorted into testing room by the  same non-disabled experimenter Experimenter explained data would be used in a large  study analyzing the way individuals viewed themselves and others Participants and confederate completed ATDP-  O, demographic questionnaire and GME Experimenter remained in the room to answer questions 
  41. 41. Instrument Validity and Reliability We do not know if the instruments were  reviewed by an expert; only talks about previous use of ATDP-O Questions raised about scale accuracy  Researchers administered test only once 
  42. 42. Internal Validity Threats   Subject characteristics  Testing / reactivity  Attitude of subjects Controlled   Instrumentation / data collector characteristics  Location
  43. 43. Conclusions Nondisabled individuals reported a more  favorable attitude toward persons with disabilities when in the presence of such an individual Habituation between nondisabled and disabled  people may be a cost-effective way to begin modifying negative attitudes Interpretations speculative -- study has many  weaknesses
  44. 44. External Validity Study can’t be generalized  Studies only people in wheelchairs and not  other disabilities Sample consists of college students from one  Southern U.S. college Results need replication 
  45. 45. Classroom Experiment
  46. 46. Experiment: Tangram Assembly Question: Does verbal communication decrease  the time it takes a group of four individuals working in tandem to assemble of specific images with tangrams? Hypothesis: Since the only thing that will change  is whether or not people are allowed to talk our hypothesis should state that verbal groups should be able to complete the images more quickly. (However, personally I think our class is full of independent thinkers who may have difficulties organizing verbally and the quiet teams may actually do better.)
  47. 47. Variables Independent: Ability to use verbal  communication (Qualitative) Dependent: Time to assemble specific  images with tangrams (Quantitative) Extraneous:   Both groups are aware of the purpose of the experiment  Some individuals may be more visual learners  Some individuals will have had more experience using tangrams  Not enough testing groups