This document summarizes a study of students changing their answers on physics problems based on peer discussion using an online peer instruction tool called DALITE. The study analyzed data from over 7,000 student-item pairs across multiple colleges. Key findings included that weaker students were more likely to change to the right answer after peer discussion, while stronger students could go either way. There were also gender differences found in first attempt performance but not after allowing answer changes. The document concludes with suggestions for future work analyzing rationales and voting data.

1. Students changing their answers, based on what
their friends say
Sameer Bhatnagar 1 Michel Desmarais 1
C. Whittaker 2 N. Lasry 3 M. Dugdale 3 K. Lenton 4 E. Charles 2
1Polytechnique Montreal
2Dawson College
3John Abbott College
4Vanier College
June 25, 2015
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 1 / 22

2. What is this all about?
Introducing a new data set
Demonstrating its potential for EDM
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 2 / 22

3. Outline
1 Glossary
2 Data
3 Results
Answer Changes
Gender Differences
Peer Voting
4 Future Work
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 3 / 22

4. Glossary
Peer Instruction
Classroom Activity popularized by Eric Mazur of Harvard
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 4 / 22

5. Glossary
Peer Instruction
Classroom Activity popularized by Eric Mazur of Harvard
DALITE
Web Based Learning Environment for Peer Instruction at Home
Let’s give the system a try!
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 4 / 22

6. Outline
1 Glossary
2 Data
3 Results
Answer Changes
Gender Differences
Peer Voting
4 Future Work
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 5 / 22

7. The Data Set: from the classroom
118 students from three different colleges
Four different teachers, five different groups
Final Grade for the course (Freshman Year Physics)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 6 / 22

8. The Data Set: from DALITE
7100 Student-item pairs
Each student item pair includes
First Answer
Rationale
Second Answer
How many votes the rationale received
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 7 / 22

9. Outline
1 Glossary
2 Data
3 Results
Answer Changes
Gender Differences
Peer Voting
4 Future Work
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 8 / 22

10. Strong students as likely to go the wrong way
0.0
0.1
0.2
0.3
0.4
bottom top
Final grade partitioned at median
P(Right−−>Wrong)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 9 / 22

11. Weak students as likely to go the right way
0.0
0.2
0.4
0.6
0.8
bottom top
Final grade partitioned at median
P(Wrong−−>Right)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 10 / 22

12. Don’t Use the Tool Alone!
Which group never talked about DALITE in class?
0.0
0.2
0.4
0.6
cw09 cw10 KJL MD NL
Different Groups
P(SwitchingAnswer)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 11 / 22

13. Gender Gap in Physics
0.4
0.6
0.8
1.0
f m
Gender
P(RightAnsweronFIRSTAttempt)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 12 / 22

14. Gender Gap in Physics except if you let them
change their minds
0.4
0.6
0.8
1.0
f m
Gender
P(RightAnsweronSECONDattempt)
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 13 / 22

15. No Gender Gap in Votes Earned
0.0
0.2
0.4
0.6
f m
Gender
AverageNumberofVotesEarned
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 14 / 22

16. Strong students do earn more votes
0.0
0.1
0.2
0.3
bottom top
Final grade partitioned at median
AverageVotesearnedoverthesemester
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 15 / 22

17. Strong, even when Wrong
0.0
0.2
0.4
0.6
bottom top
Final grade partitioned at median
AverageVotesearnedoverthesemester
forWRONGANSWER
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 16 / 22

18. Outline
1 Glossary
2 Data
3 Results
Answer Changes
Gender Differences
Peer Voting
4 Future Work
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 17 / 22

19. Future Work
NLP on Rationales
Automated Essay Scoring
Topic Modeling
Collaborative Filtering with voting data
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 18 / 22

20. Welcome to The Big Leagues
License to HarvardX
Next Step: Your edX MOOC!
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 19 / 22

21. What is this all about?
Introducing a new data set
Demonstrating its potential for EDM
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 20 / 22

22. Acknowledgements
Michel Desmarais
SALTISE research
group
PAREA
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 21 / 22

23. Thank you!
Any Questions?
Contact Info
website: sameerbhatnagar.github.io
S.Bhatnagar, M. Desmarais (Polytechnique Montreal) DALITE 22 / 22