Successfully reported this slideshow.
Your SlideShare is downloading. ×

Controlled experiments

Upcoming SlideShare
Resourcd File
Resourcd File
Loading in …3
×

Check these out next

1 of 44
1 of 44

More Related Content

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Controlled experiments

  1. 1. Controlled Experiments Tomohiro Nagashima, CC BY-NC-SA Interactive Data Science; HCII; Fall 2018 05-839: Interactive Data Science
  2. 2. A bit about me 4/4/2019 2 • Tomohiro Nagashima • Second-year PhD student in HCII • Learning sciences, instructional design, Open educational resources, learning analytics
  3. 3. The tragedy 4/4/2019 3 • https://tinyurl.com/yakdc25t
  4. 4. Learning objectives 4/4/2019 4 • Describe what an experiment is • Differentiate between experiments and observational studies • Identify different types of experiments
  5. 5. Interactive class? 4/4/2019 5 • Poll Everywhere • https://pollev.com/tnagashima649
  6. 6. Interactive class? 4/4/2019 6 https://pollev.com/tnagashima649 Do you like data science?
  7. 7. 10/30/18 7
  8. 8. Research 4/4/2019 8 Lady scientist doing experiment vector clipart. CC0. https://www.goodfreephotos.com/vector-images/lady-scientist-doing-experiment-vector-clipart.png.php
  9. 9. Quantitative vs. Qualitative Research 4/4/2019 McGill University. https://www.mcgill.ca/mqhrg/resources/what-difference-between-qualitative-and-quantitative-research 9 Quantitative research aims to test hypotheses, examine cause and effect relationships, and establish general principles using numeric data. Qualitative research aims to understand phenomena that cannot be directly measured with numbers. Research questions are usually exploratory.
  10. 10. Qualitative research: example 4/4/2019 10 Miyake, N. (1986). Constructive interaction and the iterative process of understanding. Cognitive Science 10, 151-177. https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1002_2
  11. 11. 4/4/2019 11 Miyake, N. (1986). Constructive interaction and the iterative process of understanding. Cognitive Science 10, 151-177. https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1002_2
  12. 12. Qualitative research: example 4/4/2019 12 Miyake, N. (1986). Constructive interaction and the iterative process of understanding. Cognitive Science 10, 151-177. https://onlinelibrary.wiley.com/doi/abs/10.1207/s15516709cog1002_2
  13. 13. Transforming qualitative data into quantitative 4/4/2019 13 Coding interview data
  14. 14. Experiment 4/4/2019 14 https://pollev.com/tnagashima649 Lady scientist doing experiment vector clipart. CC0. https://www.goodfreephotos.com/vector-images/lady-scientist-doing-experiment-vector-clipart.png.php
  15. 15. 10/30/18 15
  16. 16. What is an experiment? Observational Studies: we observe individuals and measure variables of interest but do not attempt to influence the responses. Experimental Studies: we deliberately impose some treatment on individuals and we observe their responses. 4/4/2019 Dalene Stangl, Experimental design concepts, Experimental Design for Behavioral and Social Sciences 16
  17. 17. Example of an observational study Recruit participants for a study. Give them journals to record hour by hour their activities the following day, including when they watch TV and when they consume snacks. Determine if snack consumption is higher during TV times. 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 17
  18. 18. Example of an experimental study Recruit participants for a study. Half of the individuals sit in a waiting room with snacks available and a TV on. The other half sit in a waiting room with snacks available and no TV, just magazines. Researchers determine whether people consume more snacks in the TV setting. 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 18
  19. 19. Exercise 1 • https://pollev.com/tnagashima649 A delivery company that owns a fleet of trucks performs a study to determine whether rotating tires at specific intervals has any effect on the number of miles a set of tires lasts. Observational study or experiment? 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 19
  20. 20. 10/30/18 20
  21. 21. Exercise 2 • https://pollev.com/tnagashima649 Researchers compare the rates of autism for children who did and did not receive the standard measles- mumps-rubella vaccine, to see if the vaccine might be responsible for autism in some children. Observational study or experiment? 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 21
  22. 22. 10/30/18 22
  23. 23. Exercise 3 • https://pollev.com/tnagashima649 Researchers classified pregnant women as being nondrinkers or light, moderate, or heavy drinkers; they examine the weights of their children at regular age intervals to see if alcohol during pregnancy results in poor growth. Observational study or experiment? 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 23
  24. 24. 10/30/18 24
  25. 25. Exercise 3 • https://pollev.com/tnagashima649 Researchers classified pregnant women as being nondrinkers or light, moderate, or heavy drinkers; they examine the weights of their children at regular age intervals to see if alcohol during pregnancy results in poor growth. Observational study or experiment? 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 25
  26. 26. Independent and dependent variables • Independent variable ( explanatory variable) • A variable that claims to explain, predict, or affect the response • Dependent variable (response variable) • Outcome of the study 4/4/2019 26Independent variable Dependent variable OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/
  27. 27. Independent and dependent variables • Exercise 4 • https://pollev.com/tnagashima649 Are the smoking habits of a person (yes, no) related to the person's gender? – Is the gender the independent or dependent variable? 4/4/2019 27 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/
  28. 28. 10/30/18 28
  29. 29. Independent and dependent variables • Exercise 5 • https://pollev.com/tnagashima649 Can you predict a person's favorite type of music (classical, rock, jazz) based on his/her IQ level? -- Is favorite type of music the independent or dependent variable? 4/4/2019 29 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/
  30. 30. 10/30/18 30
  31. 31. True experiment • Researcher-manipulated variable • Independent (explanatory) variable and dependent (response) variable • Control condition and experimental condition • no intervention, works as a baseline group • Random assignment • Each participant has an equal chance of being assigned to each condition. 4/4/2019 Grand Canyon University, Types of experimental design: https://cirt.gcu.edu/research/developmentresources/research_ready/experimental/design_types 31
  32. 32. Types of study designs • Pre-experimental • No randomization, no control group • Experimental • Random assignment of participants, treatment and control groups • Quasi-experimental • Alternative to experiments • Ex-post facto • After the “treatment” 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 32
  33. 33. Types of study designs • Pre-experimental • No randomization • No control group • No causal inference can be made 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 33
  34. 34. Types of study designs • Experimental • Random assignment of participants • Treatment and control groups 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 34
  35. 35. Types of study designs • Experimental – adding a pretest 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 35
  36. 36. Types of study designs • Experimental – two or more independent variables 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 36
  37. 37. Types of study designs • Experimental – two or more independent variables 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 37 Low-performing students High-performing students Discovery learning A B Direct instruction C D
  38. 38. Types of study designs • Quasi-experimental • When it’s hard to do randomization 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 38
  39. 39. Types of study designs • Ex-post facto • After the treatment has already been “administered” 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 39
  40. 40. Exercise 6 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 40 • Pre-experimental • No randomization, no control group • Experimental • Random assignment of participants, treatment and control groups • Quasi-experimental • Alternative to experiments • Ex-post facto • After the “treatment” • https://pollev.com/tnagashima649
  41. 41. 10/30/18 41
  42. 42. Exercise 7 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 42 • Pre-experimental • No randomization, no control group • Experimental • Random assignment of participants, treatment and control groups • Quasi-experimental • Alternative to experiments • Ex-post facto • After the “treatment” • https://pollev.com/tnagashima649
  43. 43. 10/30/18 43
  44. 44. Study design activity 4/4/2019 OLI at Stanford University, CC-BY-NC-SA: http://oli.stanford.edu/ 44 https://tinyurl.com/ydbk3y29

Editor's Notes

  • Do you like data science?
    https://www.polleverywhere.com/multiple_choice_polls/VGOgzlE4P9Xq3eX
  • These are not distinctive at all

    Case study, ethnography, grounded theory
  • One word that describes an experiment ?
    https://www.polleverywhere.com/free_text_polls/f36Xgjsiib5hSQO
  • It is important to difefrenciate between these two
  • 1. A delivery company that owns a fleet of trucks performs a study to determine whether rotating tires at specific intervals has any effect on the number of miles a set of tires lasts.
    https://www.polleverywhere.com/multiple_choice_polls/fjlf96IpuHZ1nvO
  • 2. Researchers compare the rates of autism for children who did and did not receive the standard measles-mumps-rubella vaccine, to see if the vaccine might be responsible for autism in some children.
    https://www.polleverywhere.com/multiple_choice_polls/JKQrL0z0PBsUa9X
  • 3. Researchers classified pregnant women as being nondrinkers or light, moderate, or heavy drinkers; they examine the weights of their children at regular age intervals to see if alcohol during pregnancy results in poor growth.
    https://www.polleverywhere.com/multiple_choice_polls/gjyBAD2VA6noMZD
  • Hypothesis testing?
  • Hypothesis testing?
  • 4. Are the smoking habits of a person (yes, no) related to the person's gender ? -- Is the gender the independent or dependent variable?
    https://www.polleverywhere.com/multiple_choice_polls/vEBdAONd0bLL96V
  • Hypothesis testing?
  • 5. Can you predict a person's favorite type of music (classical, rock, jazz) based on his/her IQ level ? -- Is favorite type of music the independent or dependent variable?
    https://www.polleverywhere.com/multiple_choice_polls/82H9sbmLGwUWWcb
  • Hypothesis testing?
  • Example: effect of the use of new hair care product
  • Example: discovery learning vs. direct instruction
  • Example: discovery learning vs. direct instruction
    Add pretest to make sure that the difference between the groups is not significant before the experiment. Also it allows us to measure the change from pre to post test (learning gain)
  • Example: Direct instruction vs Discovery learning and low-performing vs high-performing students
  • Example: Direct instruction vs Discovery learning and low-performing vs high-performing students
  • Direct instruction and Discovery learning
    - Different school/classroom assigned to different groups
  • Relationship between children’s exposure to violence on TV and their violent behaviors
  • 5. In a study of the relationship between mathematics achievement and attitude toward mathematics, a researcher collected two measures on all freshmen enrolled in a major university. All subjects completed a scale measuring attitudes toward mathematics during an orientation meeting. In addition, the scores for these students on the quantitative section of the SAT were collected.
    https://www.polleverywhere.com/multiple_choice_polls/Uhu96ggqgWyDlBu
  • 7. In a study of the effects of marijuana use on driving skills, college students were randomly assigned to smoke two marijuana cigarettes in the laboratory or to smoke two placebo cigarettes that taste and smell like marijuana, but do not contain the drug. A short time late, subjects took a driving test in a simulator. Which of the following best describes the research design of this study?
    https://www.polleverywhere.com/multiple_choice_polls/fsWiPUAdBxGWfD4

×