How can Cognitive Science improve Online Learning & Education?


Published on

Slides from Google Tech Talk by Joseph Jay Williams. The video presentation is on Youtube:

Title: How can Cognitive Science improve Online Learning at Google and Google in Education?

Abstract: Knowledge and technology that maximizes human learning has financial value for Google in customer education and internal training, as well as social value for the public initiatives of Google in Education. Recent research in Cognitive Science provides complementary insights to those gained from practical experience and the research in Computer Science, Education and other Learning Sciences. This talk considers how learning can be improved by: (1) Asking questions and requesting explanations; (2) Presenting specific examples to illustrate abstract principles; (3) Using tests as pedagogical rather than assessment tools. Moreover, online education provides the unique opportunity of hybrid research that is simultaneously applied and academic. Online environments satisfy the scientific requirements of randomized experiments and precise control, as well as the practical need for ecological validity, fidelity, and scalable dissemination. The Cognitive Science focus on identifying both similarities and differences across learning contexts positions it well for doing research that simultaneously advances public education and a corporate mission. In addition to presenting ongoing research at Khan Academy and MOOCs like EdX, I discuss how analogous principles can be explored in teaching end-users Google Power Search, internal training, and customer education.

Bio: Joseph Jay Williams (For resources on Cognitive Science, Online Education, Ed-Tech see:,, or does Cognitive Science research on how generating explanations promotes learning, and Online Education work on improving learning from mathematics exercises (Khan Academy), increasing motivation to learn by changing people's beliefs about intelligence (Project for Education Research that Scales:, teaching metacognitive & learning strategies in Massive Open Online Courses (EdX), and using technology to change educational and health habits. He is finishing his PhD in Psychology at UC Berkeley, and also has interests in consulting for corporate e-learning and training, web development for online education, using journalism to disseminate research to practitioners, and education in online search and problem-solving for students and entrepreneurs.

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • From Trinidad.Websites:Participation QUESTION: Where does you knowledge about learning & education come from? Student’s strategies? Teacher’s approach? Instructional designer. Policy maker?
  • Bucket model of the mind“Instructionism”Integrating into webpage.EXAMPLES: Khan Academy. MOOC on power search.Learning to program, about product specifications, sales strategies, accounting procedures, management skills.
  • Normally, we absorb content.But actively processing should help, you have probably had the experience of understanding something better after explaining it to someone else, why a solution is correct – trying to teach is the best way to learn. Extensive evidence for this.Why? Why does explaining why help learning?How do you know when to ask learners for explanations?A question I’ve explored.Two views. General boost –pay attention & spend more time, more motivated. Or Selective. Drive people to discover principles.Many studies in real-world, hard to understand, so we did the first lab study with artificial materials. Image of robots. Explaining behavior.Learning about statistics concepts like variability.In lab or online, adult participants learn material.Asked a “why?” question. E.g. Explain category membership. Explain why someone behaved in a certain way. Explain the right answer.Control: Matched it for time. Choose any strategy, describe, write out or say aloud their thoughts.Measure learning. Why is that an X?Why does Y do…?Why is Z the right solution?
  • TL: Add figure showing data for consistent items so that you have two matched figures, one with anomaly (showing this effect) and one for consistent, which I presume will be flat
  • Findings: Discovery of principles.But depends on their knowledge.Can *hurt* learning.Choose cases carefully.
  • All agree it’s extremely important to teach learning strategies, ability to learn. But how successful? Transfer is very rare.Assessments are hard.Excellent precedent to go for ambitious measure like grades – it’s cutting edge, game-changing research, on a whole new level. Practical and policy context – teachers will adapt educational technologies that help them meet their goals.Workplace environments, have to look for similar measures – e.g. Do people put in more hours, more ready to ask for advice, to report errors?Improve learning strategies – does sales performance increase, less training required, fewer hours debugging?*Go for MOOCs, because grades & all the data already measured. Can we insert short training videos?They can get practice using it on specific content.Train people in educational habit of explaining. Extending the work I mentioned to incorporate insights from reciprocal teaching. Explain *to another person*.
  • Use learning principles to guide design of platforms, features & assessments
  • How can Cognitive Science improve Online Learning & Education?

    1. 1. How can Cognitive Scienceimprove Online Learning? Joseph Jay Williams CognitiveScience.Co/learn 1
    2. 2. Research Landscape• Qualitative analyses• Randomized Controlled Trials• Longitudinal studies• Educational Data Mining• Cognitive Science 2
    3. 3. Cognitive Science 3
    4. 4. Preview• Learning: “Adding” vs. “Integrating” information. • Before: Problem-based learning • During: Request Explanations • After: Assessments as instructional tools – Testing & Mixing• Learning that promotes learning • Motivation • Strategies • Online search• Resources: • Review of Cognitive Science with practical value: • Wiki with resources on Online Education: • Contact between researchers & practitioners: 4
    5. 5. Learning: Add vs.rare Transfer is Integrate Content Exercise Add Integrate object webpageto bucket with internet 5
    6. 6. Transfer is rare• Consider: – Scenario 1: General invading a fortress can’t use full force, part of force insufficient – Scenario 2: Doctors destroying an internal tumor can’t use strong rays, but weak rays aren’t enough• Transfer extremely low (Gick & Holyoak, 1984)• Motivated MBA students appeared to learn extremely well, but failed to transfer to face-to-face in real life (Gentner, Loewenstein, & Thompson, 2003)• Transfer is so rare that it’s not a plausible goal (Detterman, 1993) 6
    7. 7. Before: Start with Questions & Problems• Problem Based Learning (Hmelo-Silver, 2006; Needham & Begg, 1998; Schwartz, 1998) How do you…? Is it possible to…? 7
    8. 8. During: Request explanations(Williams & Lombrozo, (Williams, Lombrozo, (Williams, Walker, 2010) & Rehder, in press) Lombrozo, 2012) Why…? Free study, Think Aloud… Learning Measures 8
    9. 9. Statistics problem• Learn a university’s ranking system from examples (Schwartz & Martin, 2004, Belenky & Nokes, 2011) 85% in History 69% in Physics Min of 67%, Max 90% Class Average 79% > Min of 42%, Max 87% Class Average 65% Standard Deviation 8% Standard Deviation 3% Rule for ranking John Ranked? Tom Higher personal score 85% 69% Sarah was ranked higher. > Points above average 6% 4% > Points below maximum –5% –18% > Number of deviations 0.75 1.3 above average (Z-score) < 9
    10. 10. Accuracy change 0.5Accuracy improvement from Pre- Write Thoughts Free Study 0.4 Explain test to Post-Test 0.3 0.2 0.1 0.0 Single Anomaly Multiple Anomalies Williams, Walker, & Lombrozo, 2012 10
    11. 11. Key Findings• Explaining increased:• Discovery of principles (Williams & Lombrozo, 2010)• Use of existing knowledge (Williams & Lombrozo, 2013)• Explanation’s effect was selective:• Same or even worse memory• Impaired learning if patterns were unreliable (Williams, Lombrozo, Rehder, in press)• Similar effects in 5 year olds (Walker, Williams, Lombrozo, & Gopnik, under revision)
    12. 12. Online Mathematics Exercises• Khan Academy:• Explain why that solution is correct.• Here is another student/teacher’s explanation.• Grade both.• Rate how similar they are.
    13. 13. Benefits of explanation• Instructor guided & learner generated• Learning without feedback• Abstract principles
    14. 14. After: Use Assessments as Instructional Tools• The “Testing Effect” (Roediger & Karpicke, 2006) Immediate test: Study+Study ~= Study+TestAfter hours, days, weeks: Study+Study < Study+Test Learners claim: Study+Study > Study+Test 14
    15. 15. After: Efficient Assessments use Mixing Effect• Mixing Effect (Rohrer, 2009)• Ten Benefits of Testing (Roediger et al, 2011) 15
    16. 16. Real-World Laboratories• Support Randomized Experiments or A/B Testing• Precise delivery & control• Quantitative measures of learning• Ecological validity• Evidence-based decisions• Fidelity• Scalability• Iterative improvement 16
    17. 17. Learning that improves Learning• Motivation• Learning Strategies• Online search 17
    18. 18. Increasing motivation• Beliefs about intelligence (Dweck, 2006)• Do you agree that… – Your intelligence is something very basic about you that you can’t change very much. (Fixed Theory). – No matter how much intelligence you have, you can always change it quite a bit. (Malleable Theory).• Teach a malleable theory? 18
    19. 19. Increasing motivationRandomized Controlled Trial: Middle, High School, Community College.(Paunesku, Romero et al, 2011; 2012) 19
    20. 20. Cognitive + Social Psychology• Before: Preparatory questions• During: Explanations• After: Applying conceptCurrently applying these principles to learning a “Growth Mindset”Next directions:• Create video versions for MOOCs, Khan Academy, students• Change feedback in Khan Academy exercises 20
    21. 21. Learning Strategies• Teach learning strategies through videos in MOOCs• Transfer? (Clark & Mayer, 2011)• Assessment?• Explanation; Reciprocal Teaching (Palinscar & Brown, 1984)• Change “theory” of learning• Habit & Behavior Change (Fogg, 2002)• Technology (Rose et al, 2008) 21
    22. 22. Online search Content Exercise• Sophisticated uses• “Theory”: Fishing vs. Toolkit for Problem-solving 22
    23. 23. “Best” vs. “Big” Data• Rate the plausibility of each answer. 1234567• Predict Accuracy.• Rate similarity.• Grade explanations. 23
    24. 24. Resources• Selection of Cognitive Science research applicable to (Online) Education•• Wiki with resources on Online Education:• Contact between researchers & practitioners:• Comprehensive Wiki & Newsletter on K-12 Ed-Tech• Policy Prescriptions for schools National Center for Education and the Economy Surpassing Shanghai, by Mark Tucker• Institute of Education Sciences “evidence-based education”• E-learning in Industry & Workforce, Online Corporate Training & Development
    25. 25. Questions?