Learning Beyond the Horizon: Using
Peerwise to increase engagement
for students in transition
Eamon Costello
Mark Brown
James Brunton
Lorraine Delaney
National Institute for Digital Learning – Dublin City University
BackstoryBackstory
Backstory
Backstory
Learning online
Challenges of Teaching
Programing
• Diverse student body
• Some new to programing
• Online/Flexible learning
Some Key Principles in Learning
Programing
Feedback
Reflection
Practice
Kahneman, D. (2011). Thinking, fast and slow. Macmillan. Chicago
The Question
• How can we increase engagement?
(and create a virtuous practice circle that includes
reflection and feedback)
Feedback
Reflection
Practice
Principles of the Approach Taken
“[The] biggest effects on student learning occur when
teachers become learners of their own teaching, and when
students become their own teachers”.
(Hattie 2013, p. 22)
• Test enhanced learning (Roediger & Karpickle, 2006):
testing itself promotes recall and memorisation of
knowledge providing testing includes feedback and is
periodic
Hattie, J (2013) Visible learning: A synthesis of over 800 meta-analyses relating to
achievement. Routledge
Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning taking memory tests
improves long-term retention. Psychological science, 17(3), 249-255.
Chicago
Student Task
• Create Multiple Choice Questions on PHP Web
programming topics according to question
creation criteria
• Answer questions
• Provide feedback
• Rate your peers
• Required: number of questions to
– create,
– answer,
– and comment-on/rate
How to ask
• Purchase, H., Hamer, J., Denny, P. and Luxton-Reilly, A. (2010) The quality of a PeerWise MCQ
repository. IN: Proceedings of the Twelfth Australasian Conference on Computing Education-Volume
103. Australian Computer Society, Inc.
• Tarrant, M., Knierim, A., Hayes, S. K. and Ware, J. (2006) The frequency of item writing flaws in multiple-
choice questions used in high stakes nursing assessments. Nurse Educ.Today. vol. 26, no. 8, pp 662-671.
Many educators are not good question authors (Tarrant et. al., 2006)
But well guided students may be (Purchase et. al., 2010)
Student-friendly guide to best practice in authoring MCQ questions:
– The question is clearly stated;
– The question is error free;
– The distractors (incorrect answers) are feasible;
– The accompanying explanation is good;
– the specified answer is correct.
Peerwise
• Peer assessment
• Peerwise - online social learning environment
designed for peer assessment
Results
Requirement: Create 5 questions
• 56% of the 27 students created more than 5 questions.
Engagement
0
5
10
1 5 9 13 17 21 25
Questions Created per Student
Required
Requirement: Answer 7 questions
• 85% of the 27 students created more than 5 questions.
Engagement
7
0
50
100
1 5 9 13 17 21 25
Questions Answered per Student
Required
Engagement
Denny, P. (2013). The effect of virtual achievements on student engagement. In
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 763-
772). ACM.
Chicago
Large-scale (n > 1000) randomized,
controlled experiment (Denny, 2013)
found a significant positive effect:
• on the quantity of student
contributions
• the period of time over which
students engaged
Student Feedback
Student Feedback
“Peerwise depends on the knowledge of the person
asking the questions. As a secondary teacher a lot
of the people asking questions didn't balance the
style of the question with the type of information
required. People made questions unclear by trying
to make them too hard. Also what could be done is
if the people have two write questions in the
future. Do so as they have to write 2 easy, 2 harder
and one in depth question. This will allow them to
develop their skills. It was an interesting experience
and a nice break from coding. ”
Crowd-Sourced Assessment
Lessons Learned
• Peerwise very easy to use for students and
well received
• Promotes engagement
• Requires time thought to implement
• Tip: Start simply and iterate
Questions?
More about Peerwise:
http://peerwise.cs.auckland.ac.nz/
Me:
eamon.costello@dcu.ie

Learning Beyond the Horizon: Using Peerwise to increase engagement for students in transition

  • 1.
    Learning Beyond theHorizon: Using Peerwise to increase engagement for students in transition Eamon Costello Mark Brown James Brunton Lorraine Delaney National Institute for Digital Learning – Dublin City University
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
    Challenges of Teaching Programing •Diverse student body • Some new to programing • Online/Flexible learning
  • 7.
    Some Key Principlesin Learning Programing Feedback Reflection Practice Kahneman, D. (2011). Thinking, fast and slow. Macmillan. Chicago
  • 8.
    The Question • Howcan we increase engagement? (and create a virtuous practice circle that includes reflection and feedback) Feedback Reflection Practice
  • 9.
    Principles of theApproach Taken “[The] biggest effects on student learning occur when teachers become learners of their own teaching, and when students become their own teachers”. (Hattie 2013, p. 22) • Test enhanced learning (Roediger & Karpickle, 2006): testing itself promotes recall and memorisation of knowledge providing testing includes feedback and is periodic Hattie, J (2013) Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge Roediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning taking memory tests improves long-term retention. Psychological science, 17(3), 249-255. Chicago
  • 10.
    Student Task • CreateMultiple Choice Questions on PHP Web programming topics according to question creation criteria • Answer questions • Provide feedback • Rate your peers • Required: number of questions to – create, – answer, – and comment-on/rate
  • 11.
    How to ask •Purchase, H., Hamer, J., Denny, P. and Luxton-Reilly, A. (2010) The quality of a PeerWise MCQ repository. IN: Proceedings of the Twelfth Australasian Conference on Computing Education-Volume 103. Australian Computer Society, Inc. • Tarrant, M., Knierim, A., Hayes, S. K. and Ware, J. (2006) The frequency of item writing flaws in multiple- choice questions used in high stakes nursing assessments. Nurse Educ.Today. vol. 26, no. 8, pp 662-671. Many educators are not good question authors (Tarrant et. al., 2006) But well guided students may be (Purchase et. al., 2010) Student-friendly guide to best practice in authoring MCQ questions: – The question is clearly stated; – The question is error free; – The distractors (incorrect answers) are feasible; – The accompanying explanation is good; – the specified answer is correct.
  • 12.
    Peerwise • Peer assessment •Peerwise - online social learning environment designed for peer assessment
  • 17.
  • 18.
    Requirement: Create 5questions • 56% of the 27 students created more than 5 questions. Engagement 0 5 10 1 5 9 13 17 21 25 Questions Created per Student Required
  • 19.
    Requirement: Answer 7questions • 85% of the 27 students created more than 5 questions. Engagement 7 0 50 100 1 5 9 13 17 21 25 Questions Answered per Student Required
  • 20.
    Engagement Denny, P. (2013).The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 763- 772). ACM. Chicago Large-scale (n > 1000) randomized, controlled experiment (Denny, 2013) found a significant positive effect: • on the quantity of student contributions • the period of time over which students engaged
  • 21.
  • 22.
    Student Feedback “Peerwise dependson the knowledge of the person asking the questions. As a secondary teacher a lot of the people asking questions didn't balance the style of the question with the type of information required. People made questions unclear by trying to make them too hard. Also what could be done is if the people have two write questions in the future. Do so as they have to write 2 easy, 2 harder and one in depth question. This will allow them to develop their skills. It was an interesting experience and a nice break from coding. ”
  • 23.
  • 24.
    Lessons Learned • Peerwisevery easy to use for students and well received • Promotes engagement • Requires time thought to implement • Tip: Start simply and iterate
  • 25.

Editor's Notes

  • #5 Understanding” expresses the epistemology of a university as knowledge acquired with a sense of responsibility for how it comes to be known and with the purpose of enabling enhanced action.
  • #7 Learning a programming language can be difficult for novices. It involves learning a large amount of syntactic and semantic rules (and conventions). Novice learners may lack confidence in programming as they grapple with this new subject.
  • #8 As in many areas practice is key – in computational thinking terms tapping into our Type Two Brain