Successfully reported this slideshow.

How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments

1

Share

Loading in …3
×
1 of 29
1 of 29

More Related Content

Similar to How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments

  1. 1. How online educational resources provide novel affordances for conducting practical interventions and doing psychology experiments www.josephjaywilliams.com/experiments-online-education Joseph Jay Williams, Stanford University Lytics Lab (lytics.stanford.edu), Graduate School of Education Office of the Vice Provost of Online Learning (vpol.stanford.edu) josephjaywilliams@stanford.edu www.josephjaywilliams.com Originally from Trinidad – apologies for thick accent! 1
  2. 2. Overview • • • • • • MOOCs VS Digital Online Resources Experiments: Link Real World & Laboratory Motivational Messages Experiment Simultaneous Basic & Applied Research Ed-Surge Meetup to link Research & Practice (Wed Feb 19 @ 7) Current Directions Teaching Learning Strategies Learning Coach MOOC Experiment • MOOClets Research & Practice Group 2
  3. 3. Online Education
  4. 4. Moving from Physical to Digital World
  5. 5. Experiments: Link Real World & Laboratory • Randomized assignment • Experimental Control • Rich data • Real-world environment • Authentic activities • Practical Challenges
  6. 6. Experiment • Practical goal on Khan Academy • Extend motivation research 6
  7. 7. Motivate students to practice math • Online (Math) Exercise Motivational Message: Keep up the good work! 1. Number of Problems Practiced 2. Improve Accuracy 7
  8. 8. Add motivational messages Practice-as-usual Growth Mindset Message Remember, the more you practice the smarter you become! 8
  9. 9. Which messages to add? • • • • • • "This might be a tough problem, but we know you can do it!” Remember, the more you practice the smarter you become! Some of these problems are hard. Just do your best.” Encourage “Growth Mindset” beliefs about intelligence (Dweck, 2008) Fixed Mindset Growth Mindset 9
  10. 10. Embedded in vivo Experiment Jascha Sohl-Dickstein • Practice-as-usual • Growth Mindset Message • • "Remember, the more you practice the smarter you become.”, "Mistakes help you learn. Think hard to learn from them.” 10
  11. 11. Results: More motivated? • Growth Mindset Message > Practice-asUsual • reduction in dropout rate • increase in Number of problems practiced • increase in Number Correct • Accuracy: Problems correct/Problems attempted • Accuracy improvement 11
  12. 12. Interpreting results • Obvious? • Skepticism: Anything’s statistically signficant in a MOOC 12
  13. 13. Different motivational messages Practice-as-usual Message Growth Mindset Message Positive Remember, the more you practice the smarter you become! Some of these problems are hard. Do your best! 13
  14. 14. Does any positive message work? • Practice-as-usual • Growth Mindset Message • • "Remember, the more you practice the smarter you become.”, "Mistakes help you learn. Think hard to learn from them.” • Positive Message • • "Some of these problems are hard. Just do your best." "This might be a tough problem, but we know you can do it.” 14
  15. 15. Effects of Positive Messages? • Positive Messages not better than Practice-as-Usual! • Growth Mindset better than Positive • reduction in dropout rate • increase in Number of problems practiced • increase in Number Correct • Accuracy improvement 15
  16. 16. Next: Situationally appropriate messages • Message blast, can be ignored • Providing motivational message Justin-Time: after incorrect answer • Laboratory experiments • www.assistments.org Neil Heffernan 16
  17. 17. Simultaneous Applied & Basic Research • Start with “Product” • Constraints Randomization (Technical, use Qualtrics) Behavioral/Learning Measures Williams & Williams (2013). Using Interventions to improve Online Learning. NIPS Data Driven Education workshop. tiny.cc/experimentsonlinelearning • Consider Synthesis Williams, J.J. (2013) Improving Learning in MOOCs by Applying Cognitive Science. AIED MOOCshop & NIPS Data-Driven Workshop. tiny.cc/cognitivescienceinmoocs Reading List: www.josephjaywilliams.com/education Improving outcome of psychosocial treatments by enhancing memory and learning.(in press) Perspectives in Psychological Science. • High Investment but High Return 17
  18. 18. Multidisciplinary – Relevant Communities • • • • • Association for Psychological Science 2013. Online Learning and Psychological Science: Opportunities to integrate research and practice. Williams, Saxberg, Means, Mitros. Cognitive Science Society 2013. Online Education: A Unique Opportunity for Cognitive Scientists to Integrate Research and Practice. Williams, Renkl, Koedinger, Stamper. AERA 2014– American Education Research Association. How online resources can facilitate interdisciplinary collaboration. Williams, Thille, Trumbore, Stigler, Siemens. ICLS (under review) – International Conference for the Learning Sciences. Online & Blended Education: Opportunities and Obstacles for the Learning Sciences. Williams, Linn, Slotta, Rose, Dillenbourg. CHI 2014 – Computer Human Interaction. Learning Innovations at Scale. Williams, Kizilcec, Russell, Klemmer. • • • E-learning guild’s DevLearn Educational Data Mining Artificial Intelligence in Education • • • Computer Supported Collaborative Learning Professional & Organizational Development Network American Society for Training & Development
  19. 19. Laboratory Paradigms & Online Products • A matching problem • www.edsurge.com, www.edudemic.com • josephjaywilliams@stanford.edu • Williams, J.J., Teachman, B.A., Richland, L., Brady, S.T, & Aleahmad, T. (2014). Leveraging the Internet to do Laboratory Research in the Real World. Symposium to be conducted at the annual convention of the Association for Psychological Science. San Francisco, CA. 19
  20. 20. Example: Explanation & Learning • Williams & Lombrozo (2010, Cognitive Science; 2013, Cognitive Psychology; w Rehder, JEP: General) Glorp Right Drent Wrong 20
  21. 21. EdSurge Meetup Wed Feb 19 7 pm CERAS • “Improving Ed-Tech using Research on Learning & Advancing Research on Learning using Ed-Tech” • Ed-Tech Startup Survey • Researcher Survey • Featured Company & Researcher • Panelist & Question suggestions? 21
  22. 22. Completely New Horizons for Experimentation • “Bridging Studies” • www.josephjaywilliams.com/experime nts-online-education • Williams, J. J. (2013). Enhancing Educational Research & Practice using Experiments on Online Educational Resources. Talk at Pittsburgh Science of Learning Center LearnLab Summer School, Pittsburgh, PA. tiny.cc/onlineexperiments 22
  23. 23. Next: Teaching Learning Strategies • Limited teaching time: Specific content or general strategies? • Online Advantage: Easier to run RCTs & collect longitudinal data Reinforce new educational habits & behaviors • Teach metacognitive self-questioning strategies (Palinscar & Brown, 1984, Cognition and instruction; McNamara, 2004, Discourse Processes; Williams & Lombrozo, 2010, Cognitive Science) 23
  24. 24. Drop-down text about strategies Clickable link. [Click here to learn about the “What? Why? How?” strategy]
  25. 25. Learning Coach • Tiny.cc/kalearningcoach 25
  26. 26. MOOC experiment • Motivational Video Control Energetic Thank you & Encouragement Growth Mindset (by Instructor) Growth Mindset (outside Instructor) “Belonging” • Questions for reflection after video None VS Just Think VS Write response 26
  27. 27. MOOClets Group – Develop Digital Modules • • • • • • • • • E.g. MOOClet: Math exercise, 5 minute interactive video, 30 minute lesson, Study Strategy coach, Quiz/Assessment Instructors can use for blended classroom education (High School, Community College, University) Continuously in use and being improved Easy to build, prototype, test & iterate Researchers directly implement & collect data Small scale & online access facilitates multidisciplinary collaboration & research-practitioner development (see AERA 2014 invited panel) Illustrate/Operationalize instructional design principles MOOClet Reading & Discussion Group meets Tue 11-12 PST @ Barnum (Stanford) or tiny.cc/lyticshangout Multidisciplinary group of researchers & practitioners (tiny.cc/mooclet) 27
  28. 28. Review • • • • • • • MOOCs VS Digital Online Resources Experiments: Link Real World & Laboratory Motivational Messages Experiment Simultaneous Basic & Applied Research Ed-Surge Meetup to link Research & Practice (Wed Feb 19 @ 7) Current Directions Teaching Learning Strategies Learning Coach MOOC Experiment MOOClets Research & Practice Group Meets Tuesdays 11-12 PST (tiny.cc/lyticshangout, tiny.cc/mooclet) 28
  29. 29. Acknowledgements • • • • • • Jascha Sohl-Dickstein Jace Kohlmeier & Khan Academy PERTS Sam Maldonado Lytics Lab (lytics.stanford.edu) VPOL (Vice Provost of Online Learning, online.stanford.edu) 29

Editor's Notes

  • Originally from Trinidad. I’m a Research Fellow in theLytics Lab & Office of Online Learning. Cognitive Science background – experimental psychology, statistical modeling and machine learning.Core interest is being a knowledge broker – reviewing and synthesizing the thousands of published studies in cognitive science, education research, the learning sciences to see which ones are relevant to real-world outcomes, and building products or conducting experiments that improve practically ad financially valuable outcomes. Illustrate this approach with an experiment that shows how to improve students’ motivation while learning from mathematics problems on Khan Academy.
  • o.320.27
  • ×