Peerwise - Paul Denny - Edinburgh 2011 (part 2)

  • 637 views
Uploaded on

Part 2 of Paul Denny's presentation at the LTKB workshop, Edinburgh 2011. PeerWise is a web-based repository of MCQs built by students. Students are given the responsibility of creating and moderating …

Part 2 of Paul Denny's presentation at the LTKB workshop, Edinburgh 2011. PeerWise is a web-based repository of MCQs built by students. Students are given the responsibility of creating and moderating the resource. By leveraging the creativity and energy of a class, a large, diverse and rich resource can result.

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
637
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
5
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. University of EdinburghPeer Feedback and Assessment for Science and Engineering17th December, 2010 PeerWise familiarity breeds content student-authored questions using PeerWise Paul Denny Department of Computer Science The University of Auckland New Zealand
  • 2. Goals• Hands-on with PeerWise – authoring, answering, evaluating• Selected results• Getting started – with your students
  • 3. What is PeerWise?• Web-based MCQ repository built by students
  • 4. Your turn Google: peerwise orhttp://peerwise.cs.auckland.ac.nz
  • 5. RegistrationClick the “Registration” link
  • 6. RegistrationBegin registration
  • 7. Choose a user name Enter any user name you like
  • 8. Choose a password Choose a password for your account. Make sure you remember what this is!
  • 9. Enter the “Course ID” The Course ID for this workshop is: 5279 Enter this Course ID
  • 10. Enter your “Identifier” Enter your “Identifier” value
  • 11. ConfirmYou should see a green tick. Just click“Confirm”
  • 12. You can now log in and begin...You can now log in withyour username andpassword.
  • 13. Hands-on demonstration• Creating, answering, evaluating questions
  • 14. Research• What do students think? – Auckland – Edinburgh• Repository quality?• Activity linked to performance?
  • 15. What do students think? “I actually found it enjoyable, as sad as that sounds.” Student feedback
  • 16. What do students think?• Survey responses (n = 439) – ENGGEN 131, Semester 2, 2007 Developing new questions Answering other students helped me learn questions helped me learn
  • 17. What do students think?• Survey responses (n = 387) – ENGGEN 131, Semester 2, 2009 Developing new questions Answering other students helped me learn questions helped me learn
  • 18. What do students think?• Focuses attention on learning outcomes "What I found most interesting was how other people structured their questions. It kind of made me think about what kind of topics or concepts people felt were important to the course, hence their choice of making a question on a particular topic."
  • 19. What do students think?• Focuses attention on learning outcomes• Express understanding in their own words "The biggest learning experience for me was setting up my multi-choice question... ...in the end it was a lot of help because i was just about able to answer any question that was on the same topic as my question"
  • 20. What do students think?• Focuses attention on learning outcomes• Express understanding in their own words• Question bank for drill and practice revision "I answered over 100 questions, it was a quick way to test my knowledge and if I got the answer wrong the explanations helped me learn something I wasnt too sure with."
  • 21. What do students think?• Focuses attention on learning outcomes• Express understanding in their own words• Question bank for drill and practice revision• Peer comparison "Being able to see how other people answered was great as it allowed me to recognise at which level I was at compared to everyone else"
  • 22. Edinburgh• We sought student feedback both in ‘wash- up’ sessions after the assessment and in the end of course questionnaire
  • 23. Edinburgh
  • 24. Repository quality?“The quality of questions range from meaningless questions to questions that stimulate your brain.” Student feedback
  • 25. Repository quality?• Selected finding – how often is the author’s answer incorrect?
  • 26. Repository quality?• Selected finding – how often is the author’s answer incorrect? COMPSCI 101, Semester 1, 2008 BIOCHEM 233, Semester 1, 2010 617 questions 528 questions Analysis: 10% (62 questions) Analysis: 20% (109 questions)
  • 27. Repository quality?• Selected finding – how often is the author’s answer incorrect? COMPSCI 101, Semester 1, 2008 BIOCHEM 233, Semester 1, 2010 617 questions 528 questions Analysis: 10% (62 questions) Analysis: 20% (109 questions)
  • 28. At OzBio 2010, Melbourne, October 2010Dr. Steven BottomleyFaculty of Health Sciences, Curtin University
  • 29. Edinburgh• Quality of submissions: – Average quality was very good – Few trivial questions / nonsense distracters – Highest quality questions were EXCEPTIONALLY good
  • 30. Activity linked to performance? “Using PeerWise was very useful for learning things I didnt know I didnt know.” Student feedback
  • 31. Activity linked to performance?• Do the most active students improve their position in class? – study at the University of California, San Diego prerequisite CSE 8A CSE 8B Fall 2008 Winter 2009
  • 32. Activity linked to performance?• Do the most active students improve their position in class? – study at the University of California, San Diego CSE 8A CSE 8B n = 73 Fall 2008 Winter 2009 Final exam grade provides Final exam grade provides class rank in CSE 8A class rank in CSE 8B
  • 33. Activity linked to performance?• Quartiles based on number of questions answered Q1 Q2 Q3 Q4
  • 34. Activity linked to performance?• Change in rank from CSE 8A to CSE 8B – (mean number of questions answered)
  • 35. Activity linked to performance?• Replicated the following term Winter 2009 (n=73) Spring 2009 (n=53) The highest performing The lowest performing students in the pre-requisite students in the pre-requisite course were most active course were most active
  • 36. Future work• Multi-institutional collaboration• Challenges – Calendars PeerWise – Curriculums Course 1 Course 3 Institution A Institution C Course 2 Institution B
  • 37. Future work• At Edinburgh – Does use improve course performance? – Enhancements / changes for next year • More consistent use across course • More guidance from TAs / Staff?
  • 38. Creating a new PeerWise course
  • 39. Creating a new PeerWise course 123 234 345 456 567
  • 40. Creating a new PeerWise course• Step 1)• Step 2)
  • 41. Thank you• Any questions? – Now – Later Paul Denny paul@cs.auckland.ac.nz Department of Computer Science The University of Auckland PeerWise peerwise.cs.auckland.ac.nz