Learning Beyond the Horizon: Using Peerwise to increase engagement for students in transition

1. Learning Beyond the Horizon: Using
Peerwise to increase engagement
for students in transition
Eamon Costello
Mark Brown
James Brunton
Lorraine Delaney
National Institute for Digital Learning – Dublin City University

2. BackstoryBackstory

3. Backstory

4. Backstory

5. Learning online

6. Challenges of Teaching
Programing
• Diverse student body
• Some new to programing
• Online/Flexible learning

7. Some Key Principles in Learning
Programing
Feedback
Reflection
Practice
Kahneman, D. (2011). Thinking, fast and slow. Macmillan. Chicago

8. The Question
• How can we increase engagement?
(and create a virtuous practice circle that includes
reflection and feedback)
Feedback
Reflection
Practice

9. Principles of the Approach Taken
“[The] biggest effects on student learning occur when
teachers become learners of their own teaching, and when
students become their own teachers”.
(Hattie 2013, p. 22)
• Test enhanced learning (Roediger & Karpickle, 2006):
testing itself promotes recall and memorisation of
knowledge providing testing includes feedback and is
periodic
Hattie, J (2013) Visible learning: A synthesis of over 800 meta-analyses relating to
achievement. Routledge
Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning taking memory tests
improves long-term retention. Psychological science, 17(3), 249-255.
Chicago

10. Student Task
• Create Multiple Choice Questions on PHP Web
programming topics according to question
creation criteria
• Answer questions
• Provide feedback
• Rate your peers
• Required: number of questions to
– create,
– answer,
– and comment-on/rate

11. How to ask
• Purchase, H., Hamer, J., Denny, P. and Luxton-Reilly, A. (2010) The quality of a PeerWise MCQ
repository. IN: Proceedings of the Twelfth Australasian Conference on Computing Education-Volume
103. Australian Computer Society, Inc.
• Tarrant, M., Knierim, A., Hayes, S. K. and Ware, J. (2006) The frequency of item writing flaws in multiple-
choice questions used in high stakes nursing assessments. Nurse Educ.Today. vol. 26, no. 8, pp 662-671.
Many educators are not good question authors (Tarrant et. al., 2006)
But well guided students may be (Purchase et. al., 2010)
Student-friendly guide to best practice in authoring MCQ questions:
– The question is clearly stated;
– The question is error free;
– The distractors (incorrect answers) are feasible;
– The accompanying explanation is good;
– the specified answer is correct.

12. Peerwise
• Peer assessment
• Peerwise - online social learning environment
designed for peer assessment

17. Results

18. Requirement: Create 5 questions
• 56% of the 27 students created more than 5 questions.
Engagement
0
5
10
1 5 9 13 17 21 25
Questions Created per Student
Required

19. Requirement: Answer 7 questions
• 85% of the 27 students created more than 5 questions.
Engagement
7
0
50
100
1 5 9 13 17 21 25
Questions Answered per Student
Required

20. Engagement
Denny, P. (2013). The effect of virtual achievements on student engagement. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 763-
772). ACM.
Chicago
Large-scale (n > 1000) randomized,
controlled experiment (Denny, 2013)
found a significant positive effect:
• on the quantity of student
contributions
• the period of time over which
students engaged

21. Student Feedback

22. Student Feedback
“Peerwise depends on the knowledge of the person
asking the questions. As a secondary teacher a lot
of the people asking questions didn't balance the
style of the question with the type of information
required. People made questions unclear by trying
to make them too hard. Also what could be done is
if the people have two write questions in the
future. Do so as they have to write 2 easy, 2 harder
and one in depth question. This will allow them to
develop their skills. It was an interesting experience
and a nice break from coding. ”

23. Crowd-Sourced Assessment

24. Lessons Learned
• Peerwise very easy to use for students and
well received
• Promotes engagement
• Requires time thought to implement
• Tip: Start simply and iterate

25. Questions?
More about Peerwise:
http://peerwise.cs.auckland.ac.nz/
Me:
eamon.costello@dcu.ie