Peerwise and students’ contribution experiences from the field

  • 91 views
Uploaded on

 

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
91
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • http://peerwise.cs.auckland.ac.nz/docs/students/uoa/#scoring

Transcript

  • 1. Lenandlar Singh Department of Computer Science University of Guyana lenandlar.singh@uog.edu.gy Peerwise and Students’ Contribution: Experiences from the Field CO13 – Connecting Online Conference February 03, 2013
  • 2. Outline  Contributing Student Pedagogy (CSP)  PeerWise and CSP  PeerWise – the Tool  Brief Literature Review  Experiment  Results  Observations/Discussion  Future Work  References
  • 3. Contributing Student Pedagogy(CSP)  “…a pedagogy that encourages students to contribute to the learning of others and to value the contributions of others”  “CSP in formal education is anticipatory of learning processes found in industry and research, in which the roles and responsibilities of ‘teacher’ and ‘student’ are fluid”  “Preparing students for this shift is one motivation for use of CSP”  “CSP approaches are linked to constructivist and community theories of learning, and provide opportunities to engage students more deeply in subject material” [Hamer et al., 2008]
  • 4. Contributing Student Pedagogy(CSP)  Traditionally, students are much more familiar with answering questions set by their course instructor than they are with authoring questions of their own [Sykes et al., 2011]  benefits of the latter is well documented [Nicol, 2007]  “By requiring students to engage in new roles, deeper and more substantial learning may emerge” [Hamer et al., 2008]  “…professional computing practice has largely diverged from hierarchical structures and now operates in a style markedly different from traditional formal education in which an authority disseminates textbook knowledge” [Hamer et al., 2008]  Students now produce much more context than before – Youtube Videos, Social Networks, etc… [Denny et al., 2010a]  use of technology makes it possible for students to efficiently share their authored questions with one another.
  • 5. PeerWise and CSP  PeerWise is a web-based system that allows for the development multiple choice question banks entirely by students [Denny, 2008]  Students Create MC questions, provide answers and explanations  Students can answer MC questions, provide comments, rate questions  Give Students a rating score for contribution  Developed in 2007 at University of Auckland, NZ by Paul Denny [PhD Research]
  • 6. Why PeerWise?  Many tools available for creating questions – MC, Open-response, etc [Sykes et al., 2011]  PeerWise “attempts to exploit the familiarity that students have with social software and the phenomenon of Web 2.0” [Denny, 2010]  “…familiarity of students with other social networking tools, and PeerWise’s intuitive interface, facilitates ease of use by students with a range of computing experience” [Sykes et al., 2011]
  • 7. Literature Review  Student Perceive PeerWise to be useful and Enjoyable [Denny et al., 2008]  external motivators are needed only for question generation; exam performance is correlated with participation in on-line discussions; and, despite student enthusiasm, drill-and-practice use does not contribute to exam success[Denny et al., 2008b]  Measure benefits to student Learning [Denny et al., 2010b]
  • 8. Literature Review  Student who used PeerWise performed significantly better in Final Exam compared to low activity users; was easy to set up by instructor, and students used it to study for final exam [Denny et al., 2010c]  Student reported positively about PeerWise with explicit recognition of its value to learning [Sykes et al., 2011]  Student contributed beyond what was expected and performance on peerwise was correlated significantly with exam scores[Sykes et al., 2011]  Students are eager participants, rate PeerWise highly, and produce large repository of relevant, good quality questions [Bottomley & Denny, 2011]
  • 9. Literature Review – Subject Domains  Computer Science/Programming - [Denny, 2008]  Engineering [Denny et al., 2009]  Biomedical Engineering [Denny et al., 2010a]  Information Systems [Smith et al., 2011]  Veterinary Medicine [Sykes et al., 2011]  Biomedical Science [Bottomley & Denny, 2011]  Introductory Physics [Bates et al., 2011]  Introductory Cell Biology [Smith, 2011]
  • 10. PeerWise – The Tool
  • 11. PeerWise
  • 12. Social PeerWise
  • 13. Social PeerWise
  • 14. PeerWise Uptake UG
  • 15. PeerWise at UG (Dept. of CS)
  • 16. PeerWise at UG (Dept. of CS)
  • 17. Experiment (2011 & 2012)  CSI 312 (2011) & CSI 3102 (2012) – Introduction to Object Oriented Programming  Approximately:  25 3rd year BSc in Comp Sci students (2011)  30 3rd year BSc in Comp Sci students (2012)  Peerwise used for Approximately 8 weeks in both years  Students were asked to created 3-5 MCQs, and answer, comment, and participate as much otherwise  2% of Courswork Marks allocated (2011) & 3% (2012)  No training on creating MC questions was conducted
  • 18. Administrator’s Interface
  • 19. Results – Contribution Summary
  • 20. Results
  • 21. Results – Contribution Breakdown View Details
  • 22. Results – Individual Contribution
  • 23. Results – Individual Contribution
  • 24. Results – Individual Contribution
  • 25. Results – Sample Comments Question Details
  • 26. Performance Analysis Metric Total Average/Student MCQs Written 126 7.4 Answered 894 52 Comments 342 20 Answered Correctly 693(78%) 41 Answered Incorrectly 201(22%) 12 Metric Total Average/Student MCQs Written 115 4.0 Answered 666 23 Comments 61 2 Answered Correctly 415(62%) 15 Answered Incorrectly 251(38%) 9
  • 27. Usage Pattern – CSI 312/2011
  • 28. Usage Pattern – CSI 3102/2012
  • 29. PeerWise Contribtion vs Final Exam Performance  Correlations Answered Commented AnsCorrect Final Exam Marks Created 0.4 0.3 0.63 0.41 0.4 0.33 -0.86 (0.25) Answered -0.167 0.05 Commented 0.410 0.60 0.47 0.67 -.507 0.31 AnsCorrect 0.98 0.95 -.188 0.15
  • 30. Student Survey  Students were asked a number of questions via an online survey about their experience using Peerwise:  Ease of use  Impacted their learning  Motivation to use tool and contribute  If believe tool is useful for students Students Response “Peerwise was an awesome tool once I got acquainted with it (not that it took much time to get acquainted with).”
  • 31. Student Survey Students Response cont’d “In choosing question content, it caused me to do research on particular areas before I posted a question. I had to be sure about something before I posted it” “ Answering and reviewing other person's questions also taught me alot as it was important to research and be certain before presenting suggestions and reviews to questions.”
  • 32. Student Survey Students Response cont’d “I believe the interface was easy to use and presented a useful way of keeping my interest(the badges and leader board). In fact, once I got started, seeing my position rise on the leader board and getting new badges was addictive.” “I definitely think the tool would be useful for students. My only regret is that more students didn't post and critique questions posted.”
  • 33. Student Survey Students Response cont’d “My recommendation is that even more use of this tool be advocated for the next batch of students. Also, I wish more emphasis could be placed on them getting started earlier. I found it fun and think that persons will enjoy it once they get started.” “I would also advocate its use for other subject areas, it's definitely a great learning tool.”
  • 34. Student Survey Students Response cont’d “Setting up questions was easy enough.” “…the ability to add explanations and comments for questions was a big help in instances where I may have answered a question incorrectly” “ a contributor should be notified maybe by email when a question that they supplied has comments especially in cases where the comments may be critical of the answer the contributor would have chosen”
  • 35. Student Survey Students Response cont’d “ I actually used the tool in preparation for my exams, i wasn't able to get through all of the questions however. I think it can be useful to students as a quick reference and a quick crash study before exams in order to sharpen up before.” “ “On the flip side questions that are incorrect need to be fixed so as to ensure students don't end up taking the wrong answers and thinking those are correct.”
  • 36. Student Survey Students Response cont’d “it was useful because it encompasses a new way of learning” “its a very useful tool for student because it has a new method of learning whereas the student post questions, also provide answer and explanation.” “The idea of class participation motivated me whereas everyone is involved. Also both contributing and answering question motivated me.” “It was very easy, no complication at all”
  • 37. Student Survey Students Response cont’d “Well to my knowledge Ive learn a lot whereas the class is involved and everyone will try to compete to post "hard" questions, by doing that i have leaned a lot.” “I think it should be implemented for other courses especially in Computer Science”
  • 38. Observation/Discussion  Students contributed more than expected  Some evidence that ‘low-achieving’ students were more active in creating and answering questions  Negative/low correlation between number of questions created and performance on final exam  Negative/low correlation between all other variables and Final Exam performance
  • 39. Conclusion  Did it work?  Yes, for 2%/3% most definitely  Students enjoyed working with the tool
  • 40. Future Work  Further Analysis on Existing Dataset  Use Taxonomy to categorize/rate/rank questions  Develop/Use Taxonomy to Evaluate Questions  Use PeerWise extensively – Difference Groups/Courses
  • 41. References  Bates S.P., Galloway R.K., McBride K.L., (2011) Student generated content: using PeerWise to enhance engagement and outcomes in introductory physics courses. Proceedings of the 2011 Physics Education Research Conference  Bottomley, S., & Denny, P. (2011). A Participatory Learning Approach to Biochemistry Using student Authored and Evaluated Multiple-choice Questions. Bi0chemistry and Molecular Biology Education, 39(5), 352-361.  Denny,P., Hamer, J., Luxton-Reilly, A., & Purchase, H. (2008). PeerWise. Koli Calling ’08 November 13-16, 2008, Koli, Finland  Denny, P., Luxton-Reilly,A., & Hamer, J.(2008b). Student Use of the PeerWise System. ITiCSE’08, June 30–July 2, 2008, Madrid, Spain.  Denny, P., Hamer, J., & Luxton-Reilly,A.(2009). Students sharing and evaluating MCQs in a large first year Engineering course . 20th Australasian Association for Engineering Education Conference , Adelaide, Australia.  Denny, P. (2010). Motivating Online Collaborative Learning. ITiCSE’10, June 26–30, 2010, Bikent, Ankara, Turkey  Denny, P., Micou, M., & Simon, B.(2010a). Evaluation of PeerWise as an Educational Tool for Bioengineers. American Society for Engineering Education.
  • 42. References Denny, P., Hanks, B., & Simon, B., (2010b). Peerwise: replication study of a student-collaborative self-testing web service in a U.S. setting. SIGCSE 2010. ACM, USA, pp.421-425.  Denny, P., et al. (2010c). Evaluation of PeerWise as an Educational Tool for Bioengineers. American Society for Engineering Education, 2010  Hamer,J. et al. (2008). Contributing Student Pedagogy. Inroads -Sigcse Bulletin, 40(4), 194-212.  Hooper, A.S.C., Park, S.J., & Gerondis, G. (2011). Promoting student participation and collaborative learning in a large INFO 101 class: Student perceptions of PeerWise Web 2.0 technology. Proceedings of Higher Education Research and Development Society of Australasia International Conference (HERDSA 2011).  Nicol, D., 2007. E-assessment by design: using multiple-choice tests to good effect. Journal of Further and Higher Education, 31 (1) pp.53-64.  Smith, K.M. (2011). Peerwise Participation –supporting students in a multiple choice environment. Proceedings of the Effective Learning in the Biosciences Conference, Edinburgh, UK.  Sykes, A., et al. (2011). PeerWise - The Marmite of Veterinary Student Learning