James gaynor t&l conference presentation-v4


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Online collaborative tool – group work Applications – distance learning/online degrees New method
  • Sadler and good: Research questions: Student-grading as a substitute for teacher grades and Student-grading as a pedagogic tool for student learning . Outcomes: In this study, self-grading appears to further student understanding of the subject matter being taught. AND Students should be trained to grade accurately and be rewarded for doing so ( Teachers should be aware that lower performing students tend to inflate their own grades, whereas the grades of higher performing students may suffer when graded by others. Teachers should monitor the accuracy of student grades and not simply assume that grades are awarded fairly by all students). - Also obvious benefits of saving teaching time (and hopefully teachers using time more effectively) • Signature Block: Any piece of work is signed by the group. The group and/or individual can refuse to sign, which means they don’t receive marks • Effort marks: Various methods where a mark is given to each student by their peer, as either a grade, number or percentage. • Bonus marks: Students can award their peers bonus marks for extra effort. Give list of various categories (leadership, organisation, data collection, etc.) and you then put +2 to -2 on effort, and the average is taken away from the average group mark.
  • Subjective assessment is a form of questioning which may have more than one correct answer (or more than one way of expressing the correct answer). There are various types of objective and subjective questions. Objective question types include true/false answers, multiple choice, multiple-response and matching questions. Subjective questions include extended-response questions and essays. Objective assessment is well suited to the increasingly popular computerized or online assessment format.
  • - For 2010-11, there were six contributions towards marks (Table), half of which were a group mark, half an individual - Last year, there were two contributions towards marks. Academic judgement of contribution (B) and Academic judgement of the sites (E).
  • >2.5 (I gave lower than students) = One student suspended studies. Other was certainly an outlier – thought highly of by peers but this did not come across in my judgement >2-2.5 (I gave higher than students) = One suspended. Other had serious mitigating circumstances which other students would not have known about but would have affected her effort/very quiet student. Third was just lazy (so students gave low mark), but knows how to play the system so that I’d give him a high mark >1.5 – 2 (One of each) = The student I undermarked compared to PA is a overseas student with relatively poor english. Her contribution did not come across well – prefferred to work in Word and transfer it to a wiki. The student I overmarked must have been a genuine anomoly.
  • Figure: Representation of the how the average PA mark awarded to a student compared to their own self-assessment. Trend line is y=x: data points above trend line show students who awarded themselves higher marks than their peers, whilst data points below trend line show students who awarded themselves lower marks than their peers. 26 of the 32 students returned their PA forms so only these were considered here (only 25 points are visible, since there are two occurrences of the third lowest PA mark).
  • Figure 2: Three groups each prepared a Wiki. Academic mark (Code E, Table 2): Five academics awarded a mark out of 20, which was adjusted to be out of 35. Group PA (Code D, Table 2): 26 of the 32 students returned their PA forms, all of which distributing 30 marks between the 3 wikis. The mark was adjusted to be out of 35 for comparison purposes.
  • Evaluation: Students asked "It was very interesting having to assess my teammates and other websites during the peer assessment portion of the website task“ and "I really enjoyed working as a group whilst preparing the websites using the wiki application“ 3 agreed. 1 neither agreed or disagreed. 1 disagreed.
  • - Average of 60 might be good for when the external examiners come and visit, but might not be a true reflection. Might not necessarily be mixed ability groups.
  • James gaynor t&l conference presentation-v4

    1. 1. “ Assessing group wiki projects using peer assessment ” Teaching and Learning Conference, University of Liverpool 29 th June 2011. James Gaynor, Department of Chemistry
    2. 2. Overview <ul><li>What is a wiki? </li></ul><ul><ul><li>Wiki statistics </li></ul></ul><ul><li>Peer and self assessment </li></ul><ul><ul><li>Various modes </li></ul></ul><ul><ul><li>Why use it? </li></ul></ul><ul><li>Case study </li></ul><ul><ul><li>Introducing CHEM331 </li></ul></ul><ul><ul><li>Analysis/Results </li></ul></ul><ul><ul><li>Thoughts of students </li></ul></ul><ul><ul><li>Limitations/Improvements </li></ul></ul>
    3. 3. What is a wiki? <ul><li>Online collaborative tool – group work. </li></ul>
    4. 4. What is a wiki?
    5. 5. Wiki statistics <ul><li>Some major limitations: </li></ul><ul><li>Working in word and then transferring material over. </li></ul><ul><li>People being verbally helpful rather than actually logging in and helping. </li></ul><ul><li>Students working together from one computer. </li></ul>
    6. 6. Wiki statistics
    7. 7. Peer and self assessment <ul><li>Assessing group work can obviously be tricky. One mark for all not necessarily fair. </li></ul><ul><li>PA/SA help develop a variety of skills: </li></ul><ul><ul><li>Reflection. </li></ul></ul><ul><ul><li>“ Correcting a classmate’s work can be as much as a part of the assignment as taking the test itself .” Sadler and Good. </li></ul></ul><ul><li>Methods of PA/SA (detailed guidelines are important). </li></ul><ul><ul><li>Distribute marks, e ffort marks, s ignature block and bonus marks. </li></ul></ul><ul><ul><li>Web PA. </li></ul></ul>Sadler, Philip M. and Good, Eddie (2006) 'The Impact of Self- and Peer-Grading on Student Learning ', Educational Assessment, 11 (1), 1 — 31
    8. 8. Chemistry soft skills Hanson, S.; Overton, T. Skills required by new chemistry graduates and their development in degree programmes , Higher Education Academy, Physical Sciences Centre, 2010 “ With respect to your career since completing your undergraduate degree, whether working, training or undertaking other activities, please indicate the value of the areas of knowledge or skills listed.” % of those given as useful/very useful.
    9. 9. Case study: CHEM331 <ul><li>Final year BSc students – core organic chemistry (15 credits). </li></ul><ul><li>Biological chemistry portion contains three main areas. Students need to cover two of them in detail (self-learn). </li></ul><ul><li>Cohort (32) split into 3 groups of 10/11. </li></ul><ul><ul><li>Part 1: Prepare individual essays on an assigned topic. </li></ul></ul><ul><ul><li>Part 2: As a group, prepare a website on a different topic. </li></ul></ul><ul><li>PA/SA new for this year. </li></ul>Activity % Assessment Notes Lectures (22) 60 Examination Two hours Tutorials (3) 3 Continual Individual essay 18.5 Subjective 1500 words Group website via a wiki 18.5 Numerous (next slide) Individual and group marks awarded, both of which contained PA/SA portions
    10. 10. Case study: CHEM331 Code Assessment Procedure % How were these assessed? Individual Mark A Individual PA 25 All PA marks for an individual were averaged and adjusted to be out of 25. B Contribution 25* Same process as students used to complete PA, but using data from wiki stats, contribution at meetings (from minutes), effort on their page (not content), etc. C Individual assignment Each student had an individual task/webpage (determined by the group). The scientific content on each page marked by same assessors as essays. Group Mark D Group PA 10 All PA marks averaged and adjusted to be out of 10. E Academic Judgement 35 Five academics. Average mark given. More emphasis on presentation here since scientific content is marked above. F Meeting minutes 5 Academic.
    11. 11. Case study: CHEM331 <ul><li>Comparison of two sets of marking criteria: </li></ul><ul><li>Individual contribution: </li></ul><ul><ul><li>From PA (25%): students distributed 100 marks between their own group (including themselves). </li></ul></ul><ul><ul><li>Contribution mark (from staff, 25%*): 1 staff member distributed 100 marks between a group. </li></ul></ul><ul><li>Judgement of wiki quality: </li></ul><ul><ul><li>From PA (10%): students distributed 30 marks between all 3 groups. </li></ul></ul><ul><ul><li>Contribution mark (from staff, 35%): 6 staff members awarded a mark out of 20. </li></ul></ul><ul><ul><li>Both marks were adjusted to be out of 35 for comparison. </li></ul></ul><ul><li>Only 26/32 students submitted their peer assessment forms. </li></ul>
    12. 12. Case study: CHEM331 <ul><li>a. Individual contribution </li></ul><ul><li>Under marked: The mark given by academic judgement was lower than PA. </li></ul><ul><li>Over marked: The mark given by academic judgement was higher than PA. </li></ul><ul><li>Academic judgement marks were awarded prior to knowing PA results. </li></ul><ul><li>Average difference is 1/25 (4% error). </li></ul>
    13. 13. Case study: CHEM331 <ul><li>a. Individual contribution </li></ul>
    14. 14. Case study: CHEM331 <ul><li>b. Individual contribution </li></ul><ul><li>Weaker students gave themselves higher marks than they received from peers. Similar observations seen elsewhere (Boud, Sadler & Good). </li></ul>Boud, D. (1989). The role of self-assessment in student grading. Assessment and Evaluation in Higher Education, 14, 20–30
    15. 15. Case study: CHEM331 2. Judgement of wiki quality <ul><li>Staff marking not done via a PA method and hence there is a bigger deviations since marking criteria for students and staff different. </li></ul><ul><li>Average for both staff and PA marking was 60%. </li></ul>
    16. 16. Case study: CHEM331 <ul><li>Results/Conclusions </li></ul><ul><li>Determination of individual contribution marks by PA and academic judgement are quite consistent (1 mark difference). </li></ul><ul><li>Determination of group mark by academic judgement vs PA was less consistent – differing marking criteria. </li></ul><ul><li>For individual PA vs SA, the students who received a higher PA mark from their group often awarded themselves a lower SA mark. The converse is also true. </li></ul><ul><li>For group PA vs SA, all groups awarded their wikis higher SA marks than their peers. </li></ul><ul><li>Thoughts of students </li></ul><ul><li>Only 5 completed the evaluation. Mixed feedback. </li></ul>
    17. 17. Case study: CHEM331 <ul><li>Limitations/Improvements </li></ul><ul><li>“ Distribute marks” approach means there is a set average. </li></ul><ul><li>Not all students submitted forms – make it compulsory so that they don’t only lose their own marks, they lose all PA marks. </li></ul><ul><li>Transparency issues with PA assessment procedure. Students were told they would have to complete PA from the beginning, but were not told how they this would work until end. </li></ul><ul><li>New fees/increased internationalization. </li></ul><ul><li>Perhaps increase PA portion of marks for next year. Shall we give students their average PA marks back as feedback? Some careful thought required. </li></ul><ul><li>Need to repeat with the 2011-2012 cohort. </li></ul><ul><li>Would be useful for our MChem students to do something similar. Or introduce more group work in general. </li></ul>
    18. 18. Acknowledgments <ul><li>Organising committee. </li></ul><ul><li>Colleagues in chemistry: Nick, Richard, Rick, Andrew and Neil </li></ul><ul><li>You for your attention. </li></ul><ul><li>ANY QUESTIONS? </li></ul>