Data from experiments

Tomohiro Nagashima
Tomohiro NagashimaMA Candidate, Stanford Graduate School of Education
Data from Experiments
Tomohiro Nagashima, CC BY-NC-SA
Interactive Data Science; HCII; Fall 2018
05-839: Interactive Data Science
Study design activity
https://tinyurl.com/yaz5ghl4
- Find the submission by the person next to you
- Give comments on their study design with each other
- ~10 min
https://tinyurl.com/ybpp8p3q
- Submit the revised version of your study idea
- I want to see how it got improved!
- ~15 min
- https://tinyurl.com/y7hahbys (Tue slides available)
4/4/2019
2
Learning objectives
4/4/2019
3
• Differentiate causation from correlation
• Describe what a lurking variable is and
how it works
Correlation vs. Causation
Example: correlation
A Pennsylvania research firm conducted a study in which 30 drivers (of
ages 18 to 82 years old) were sampled, and for each one, the maximum
distance (in feet) at which he/she could read a newly designed sign was
determined. The goal of this study was to explore the relationship between
a driver's age and the maximum distance at which signs were legible,
and then use the study's findings to improve safety for older drivers.
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Data
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Data
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Scatterplot
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Scatterplot
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Scatterplot
Exercise 1 & 2: https://pollev.com/tnagashima649
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
4/4/2019
11
4/4/2019
12
Correlation does not imply causation
Number of firefighters sent Level of fire damage
Andrew Magill. “Firefighters”, CC-BY “Former Joss Cardwall Center”, CC-BY
https://www.flickr.com/photos/amagill/3225245640 https://www.geograph.ie/photo/5386305
Fire damage example
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Lurking variable
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
Exercise 3
https://pollev.com/tnagashima649
Research has suggested that there is a correlation between Internet usage
and obesity. Using this information, a fictional study found a strong positive
relationship between the number of hours of daily Internet usage and Body
Mass Index, a measure of body weight.
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
4/4/2019
17
Exercise 4
https://pollev.com/tnagashima649
The tourism industry hires a marketing company to promote overseas
vacations. The marketing company's research shows a strong positive
correlation between the number of overseas vacations booked and amount
of free time, in days, people have each year. Therefore, the marketing
company decides to develop an advertising campaign aimed specifically at
people who have lots of free time.
Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
4/4/2019
19
Correlation and Causation
https://tinyurl.com/ycmtmsbv
“Internet use in class leads to lower test scores”
http://www.digitaljournal.com/internet/internet-use-in-class-leads-to-lower-test-scores/article/482661
4/4/2019
21
Correlation and Causation
https://tinyurl.com/ycmtmsbv
“Internet use in class leads to lower test scores”
https://tinyurl.com/y92gkbv2
Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom
learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
Internet use and test scores
https://pollev.com/tnagashima649
• Pre-experimental
• No randomization, no control group
• Experimental
• Random assignment of participants, treatment and control
groups
• Quasi-experimental
• Alternative to experiments
• Ex-post facto
• After the treatment has already been “administered”
4/4/2019
24
Internet use and test scores
• Independent variable
• Internet use
• Average duration online
• Average number of requests
• Self-reported internet use
• Dependent variable
• Academic performance
• Final-exam score
Internet use and test scores
Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom
learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
Internet use and test scores
Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom
learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
Internet use and test scores
https://pollev.com/tnagashima649
What is a possible lurking variable here?
4/4/2019
29
Thank you!
https://tinyurl.com/yca5rxzj
1 of 30

Recommended

Education - K 12 STEM Education - LEAD:WNC 2014 by
Education - K 12 STEM Education - LEAD:WNC 2014Education - K 12 STEM Education - LEAD:WNC 2014
Education - K 12 STEM Education - LEAD:WNC 2014LEAD:WNC
374 views5 slides
Govt Jobs Guide 2010 by
Govt Jobs Guide 2010Govt Jobs Guide 2010
Govt Jobs Guide 2010HRGovGal
650 views2 slides
Cannabis on Campus - What are the Rules with the Federal Government by
Cannabis on Campus - What are the Rules with the Federal GovernmentCannabis on Campus - What are the Rules with the Federal Government
Cannabis on Campus - What are the Rules with the Federal GovernmentCannabis News
54 views13 slides
Information habits in continuously changing information environments : digg... by
Information habits in continuously changing information environments : digg...Information habits in continuously changing information environments : digg...
Information habits in continuously changing information environments : digg...Thomas Hapke
1.9K views61 slides
Online-Research: International Similarities and National Differences - Poland by
Online-Research: International Similarities and National Differences - PolandOnline-Research: International Similarities and National Differences - Poland
Online-Research: International Similarities and National Differences - PolandPawel Kuczma
183 views13 slides
Trinity Kings World Leadership: Family Franchising Systems: Creating A Cultur... by
Trinity Kings World Leadership: Family Franchising Systems: Creating A Cultur...Trinity Kings World Leadership: Family Franchising Systems: Creating A Cultur...
Trinity Kings World Leadership: Family Franchising Systems: Creating A Cultur...Terrell Patillo
313 views10 slides

More Related Content

Similar to Data from experiments

SIP by
SIPSIP
SIPcharlienork
444 views22 slides
SGA Campaign against Texting and Driving by
SGA Campaign against Texting and DrivingSGA Campaign against Texting and Driving
SGA Campaign against Texting and DrivingMackenzie Jarvis
127 views12 slides
Effectiveness of online advertsiment on students in zambia by
Effectiveness of online advertsiment on students in zambia Effectiveness of online advertsiment on students in zambia
Effectiveness of online advertsiment on students in zambia Meshach Chindele
170 views57 slides
Tracker workshop ALT Conference 2016 by
Tracker workshop ALT Conference 2016Tracker workshop ALT Conference 2016
Tracker workshop ALT Conference 2016Jisc
945 views21 slides
Controlled experiments by
Controlled experimentsControlled experiments
Controlled experimentsTomohiro Nagashima
123 views44 slides
Accessing The Quality Of Online Classified Websites An Empirical Study Of Th... by
Accessing The Quality Of Online Classified Websites  An Empirical Study Of Th...Accessing The Quality Of Online Classified Websites  An Empirical Study Of Th...
Accessing The Quality Of Online Classified Websites An Empirical Study Of Th...Jim Webb
2 views14 slides

Similar to Data from experiments(20)

SGA Campaign against Texting and Driving by Mackenzie Jarvis
SGA Campaign against Texting and DrivingSGA Campaign against Texting and Driving
SGA Campaign against Texting and Driving
Mackenzie Jarvis127 views
Effectiveness of online advertsiment on students in zambia by Meshach Chindele
Effectiveness of online advertsiment on students in zambia Effectiveness of online advertsiment on students in zambia
Effectiveness of online advertsiment on students in zambia
Meshach Chindele170 views
Tracker workshop ALT Conference 2016 by Jisc
Tracker workshop ALT Conference 2016Tracker workshop ALT Conference 2016
Tracker workshop ALT Conference 2016
Jisc945 views
Accessing The Quality Of Online Classified Websites An Empirical Study Of Th... by Jim Webb
Accessing The Quality Of Online Classified Websites  An Empirical Study Of Th...Accessing The Quality Of Online Classified Websites  An Empirical Study Of Th...
Accessing The Quality Of Online Classified Websites An Empirical Study Of Th...
Jim Webb2 views
The Drug Problem And The Current Justice System Essay by Lisa Brown
The Drug Problem And The Current Justice System EssayThe Drug Problem And The Current Justice System Essay
The Drug Problem And The Current Justice System Essay
Lisa Brown3 views
An Empirical Study Of University Websites by Kate Campbell
An Empirical Study Of University WebsitesAn Empirical Study Of University Websites
An Empirical Study Of University Websites
Kate Campbell2 views
Impact of e commerce on purchase behaviour of student in higher education by jpbbk
Impact of e commerce on purchase behaviour of student in higher educationImpact of e commerce on purchase behaviour of student in higher education
Impact of e commerce on purchase behaviour of student in higher education
jpbbk92 views
Impact of e commerce on purchase behaviour of student in higher education by Jai Prakash
Impact of e commerce on purchase behaviour of student in higher educationImpact of e commerce on purchase behaviour of student in higher education
Impact of e commerce on purchase behaviour of student in higher education
Jai Prakash548 views
Mba research project report by Mohit Verma
Mba research project reportMba research project report
Mba research project report
Mohit Verma3.9K views
Who, What, Why, and how computerize advancing by AkramKhan68669
Who, What, Why, and how computerize advancing Who, What, Why, and how computerize advancing
Who, What, Why, and how computerize advancing
AkramKhan686695 views
An Evaluation of Diverting Mobile Calls via VoIP While Traveling Abroad: The ... by Ibrahim Malek
An Evaluation of Diverting Mobile Calls via VoIP While Traveling Abroad: The ...An Evaluation of Diverting Mobile Calls via VoIP While Traveling Abroad: The ...
An Evaluation of Diverting Mobile Calls via VoIP While Traveling Abroad: The ...
Ibrahim Malek115 views
Human Resource Strategic Plan For The National Commission... by Alyssa Hase
Human Resource Strategic Plan For The National Commission...Human Resource Strategic Plan For The National Commission...
Human Resource Strategic Plan For The National Commission...
Alyssa Hase2 views
TrendsandFutureDirectionsinOnlineMarketingResearch.pdf by Preeti Jadhav
TrendsandFutureDirectionsinOnlineMarketingResearch.pdfTrendsandFutureDirectionsinOnlineMarketingResearch.pdf
TrendsandFutureDirectionsinOnlineMarketingResearch.pdf
Preeti Jadhav7 views
Universities Challenged - Universities Website League Table by Dan Watt
Universities Challenged - Universities Website League TableUniversities Challenged - Universities Website League Table
Universities Challenged - Universities Website League Table
Dan Watt161 views
Ethical Implications of Student Plagiarism in Myanmar by ijtsrd
Ethical Implications of Student Plagiarism in MyanmarEthical Implications of Student Plagiarism in Myanmar
Ethical Implications of Student Plagiarism in Myanmar
ijtsrd276 views
Marketing Research by Ameet Shinde
Marketing ResearchMarketing Research
Marketing Research
Ameet Shinde1.6K views

Recently uploaded

Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a Choice by
Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a ChoiceCreative Restart 2023: Atila Martins - Craft: A Necessity, Not a Choice
Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a ChoiceTaste
45 views50 slides
StudioX.pptx by
StudioX.pptxStudioX.pptx
StudioX.pptxNikhileshSathyavarap
101 views18 slides
JQUERY.pdf by
JQUERY.pdfJQUERY.pdf
JQUERY.pdfArthyR3
105 views22 slides
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdf by
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdfSTRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdf
STRATEGIC MANAGEMENT MODULE 1_UNIT1 _UNIT2.pdfDr Vijay Vishwakarma
130 views68 slides
INT-244 Topic 6b Confucianism by
INT-244 Topic 6b ConfucianismINT-244 Topic 6b Confucianism
INT-244 Topic 6b ConfucianismS Meyer
45 views77 slides
Gross Anatomy of the Liver by
Gross Anatomy of the LiverGross Anatomy of the Liver
Gross Anatomy of the Liverobaje godwin sunday
77 views12 slides

Recently uploaded(20)

Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a Choice by Taste
Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a ChoiceCreative Restart 2023: Atila Martins - Craft: A Necessity, Not a Choice
Creative Restart 2023: Atila Martins - Craft: A Necessity, Not a Choice
Taste45 views
JQUERY.pdf by ArthyR3
JQUERY.pdfJQUERY.pdf
JQUERY.pdf
ArthyR3105 views
INT-244 Topic 6b Confucianism by S Meyer
INT-244 Topic 6b ConfucianismINT-244 Topic 6b Confucianism
INT-244 Topic 6b Confucianism
S Meyer45 views
Creative Restart 2023: Leonard Savage - The Permanent Brief: Unearthing unobv... by Taste
Creative Restart 2023: Leonard Savage - The Permanent Brief: Unearthing unobv...Creative Restart 2023: Leonard Savage - The Permanent Brief: Unearthing unobv...
Creative Restart 2023: Leonard Savage - The Permanent Brief: Unearthing unobv...
Taste55 views
Career Building in AI - Technologies, Trends and Opportunities by WebStackAcademy
Career Building in AI - Technologies, Trends and OpportunitiesCareer Building in AI - Technologies, Trends and Opportunities
Career Building in AI - Technologies, Trends and Opportunities
WebStackAcademy45 views
Guess Papers ADC 1, Karachi University by Khalid Aziz
Guess Papers ADC 1, Karachi UniversityGuess Papers ADC 1, Karachi University
Guess Papers ADC 1, Karachi University
Khalid Aziz99 views
Pharmaceutical Analysis PPT (BP 102T) by yakshpharmacy009
Pharmaceutical Analysis PPT (BP 102T) Pharmaceutical Analysis PPT (BP 102T)
Pharmaceutical Analysis PPT (BP 102T)
yakshpharmacy009108 views
Retail Store Scavenger Hunt.pptx by jmurphy154
Retail Store Scavenger Hunt.pptxRetail Store Scavenger Hunt.pptx
Retail Store Scavenger Hunt.pptx
jmurphy15452 views
Monthly Information Session for MV Asterix (November) by Esquimalt MFRC
Monthly Information Session for MV Asterix (November)Monthly Information Session for MV Asterix (November)
Monthly Information Session for MV Asterix (November)
Esquimalt MFRC107 views
Narration lesson plan by TARIQ KHAN
Narration lesson planNarration lesson plan
Narration lesson plan
TARIQ KHAN75 views
Parts of Speech (1).pptx by mhkpreet001
Parts of Speech (1).pptxParts of Speech (1).pptx
Parts of Speech (1).pptx
mhkpreet00146 views

Data from experiments

  • 1. Data from Experiments Tomohiro Nagashima, CC BY-NC-SA Interactive Data Science; HCII; Fall 2018 05-839: Interactive Data Science
  • 2. Study design activity https://tinyurl.com/yaz5ghl4 - Find the submission by the person next to you - Give comments on their study design with each other - ~10 min https://tinyurl.com/ybpp8p3q - Submit the revised version of your study idea - I want to see how it got improved! - ~15 min - https://tinyurl.com/y7hahbys (Tue slides available) 4/4/2019 2
  • 3. Learning objectives 4/4/2019 3 • Differentiate causation from correlation • Describe what a lurking variable is and how it works
  • 5. Example: correlation A Pennsylvania research firm conducted a study in which 30 drivers (of ages 18 to 82 years old) were sampled, and for each one, the maximum distance (in feet) at which he/she could read a newly designed sign was determined. The goal of this study was to explore the relationship between a driver's age and the maximum distance at which signs were legible, and then use the study's findings to improve safety for older drivers. Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 6. Data Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 7. Data Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 8. Scatterplot Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 9. Scatterplot Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 10. Scatterplot Exercise 1 & 2: https://pollev.com/tnagashima649 Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 13. Correlation does not imply causation Number of firefighters sent Level of fire damage Andrew Magill. “Firefighters”, CC-BY “Former Joss Cardwall Center”, CC-BY https://www.flickr.com/photos/amagill/3225245640 https://www.geograph.ie/photo/5386305
  • 14. Fire damage example Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 15. Lurking variable Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 16. Exercise 3 https://pollev.com/tnagashima649 Research has suggested that there is a correlation between Internet usage and obesity. Using this information, a fictional study found a strong positive relationship between the number of hours of daily Internet usage and Body Mass Index, a measure of body weight. Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 18. Exercise 4 https://pollev.com/tnagashima649 The tourism industry hires a marketing company to promote overseas vacations. The marketing company's research shows a strong positive correlation between the number of overseas vacations booked and amount of free time, in days, people have each year. Therefore, the marketing company decides to develop an advertising campaign aimed specifically at people who have lots of free time. Open Learning Initiative at Stanford: oli.stanford.edu, CC-BY-NC-SA
  • 20. Correlation and Causation https://tinyurl.com/ycmtmsbv “Internet use in class leads to lower test scores” http://www.digitaljournal.com/internet/internet-use-in-class-leads-to-lower-test-scores/article/482661
  • 22. Correlation and Causation https://tinyurl.com/ycmtmsbv “Internet use in class leads to lower test scores” https://tinyurl.com/y92gkbv2 Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
  • 23. Internet use and test scores https://pollev.com/tnagashima649 • Pre-experimental • No randomization, no control group • Experimental • Random assignment of participants, treatment and control groups • Quasi-experimental • Alternative to experiments • Ex-post facto • After the treatment has already been “administered”
  • 25. Internet use and test scores • Independent variable • Internet use • Average duration online • Average number of requests • Self-reported internet use • Dependent variable • Academic performance • Final-exam score
  • 26. Internet use and test scores Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
  • 27. Internet use and test scores Ravizza, S. M., Uitvlugt, M. G., & Fenn, K. M. (2017). Logged in and zoned out: How laptop internet use relates to classroom learning. Psychological science, 28(2), 171-180. http://journals.sagepub.com/doi/abs/10.1177/0956797616677314
  • 28. Internet use and test scores https://pollev.com/tnagashima649 What is a possible lurking variable here?

Editor's Notes

  1. 1. A study examined how the percentage of participants who completed a survey is related to the monetary incentive that researchers promised to participants. The relationship is displayed in the scatterplot on the screen. What is the direction of this relationship? https://www.polleverywhere.com/multiple_choice_polls/AZGmOZOHTQh2rNe
  2. 2. In the context of this example, when researchers promised higher payments, what happened to the percentage of participants who completed the survey? https://www.polleverywhere.com/multiple_choice_polls/ho3wqYR87NMJDJe
  3. XX move this url / viz to the next deck… also need a better intro…
  4. XX move this url / viz to the next deck… also need a better intro…
  5. XX move this url / viz to the next deck… also need a better intro…
  6. 3. Research has suggested that there is a correlation between Internet usage and obesity. Using this information, a fictional study found a strong positive relationship between the number of hours of daily Internet usage and Body Mass Index, a measure of body weight. Which of the following is true? https://www.polleverywhere.com/multiple_choice_polls/3nI4jWchw5NBqXG
  7. 4. The tourism industry hires a marketing company to promote overseas vacations. The marketing company's research shows a strong positive correlation between the number of overseas vacations booked and amount of free time, in days, people have each year. Therefore, the marketing company decides to develop an advertising campaign aimed specifically at people who have lots of free time. Which of the following is true about this decision? https://www.polleverywhere.com/multiple_choice_polls/iENQaZf8kg08gCy
  8. XX move this url / viz to the next deck… also need a better intro…
  9. Is there a causal relationship between the internet use in class and academic performance? https://www.polleverywhere.com/multiple_choice_polls/jyMId2swdkVMdg7
  10. XX move this url / viz to the next deck… also need a better intro…
  11. Ex-post facto
  12. Go to the "Method" section of the article and read the information provided there. Given the information, which study design type do you think would this study be categorized into? https://www.polleverywhere.com/multiple_choice_polls/A5z4FlK8M7ba6RI
  13. Whether they are focused on the lecture / how much they’re focused on the internet or lecture / what’s on the final exam, and if it was taken from the lecture (how much content on the exam was taken from the lecture content) /
  14. What is a possible lurking variable in this study? https://www.polleverywhere.com/free_text_polls/J4W9U2BA7cu0u09
  15. Whether they are focused on the lecture / how much they’re focused on the internet or lecture / what’s on the final exam, and if it was taken from the lecture (how much content on the exam was taken from the lecture content) /