TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

300 views

Published on

Presenter: Hubert Nachtegaele
Organisation: Universiteit Antwerpen

Description: Our teaching staff asks support for types of “Self and Peer Assessment” which are not possible with the S&PA tool incorporated in Bb Learn.

Our Blackboard Support Team tries to support the desired types of S&PA by creative combinations solely of Bb Learn Course Tools, without using external tools (except Excel). In this session we will show how we realize this for different types of S&PA: “Groups reviewing assignments of other groups”, “Students assessing their peers and themselves within a group”, “Peer Reviewing using a rubric”, and "Anonymous Peer Feedback for oral presentations"

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
300
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

  1. 1. Peer Feedback methods based on Bb Learn Tools Extensions to the Bb-Learn Tool “Self and Peer Assessment”
  2. 2. Peer Feedback Methods based on Blackboard Learn Tools Peer Review with Rubrics Peer Review between Groups Peer Assessment within Groups Peer Feedback via My Grades Presentation by: Hubert Nachtegaele, Educational Technologist Christine Verclairen, Assistant Academic Staff Extensions to the Bb-Learn Tool “Self and Peer Assessment”
  3. 3. Peer Review with Rubrics Cartoon by Nick D Kim, strange-matter.net (please see site for terms of reuse) Extensions to the Bb-Learn Tool “Self and Peer Assessment” Hubert Nachtegaele, Educational Technologist Annick De Vylder, Assistant Academic Staff Prof. Dr. Jelle Boeve-de Pauw and prof. Dr. Piet Van den Bossche, Faculty of Social Sciences
  4. 4. Why? ± 100 students “Master of Instructional and Educational Sciences” Assignment: write paper, apply/show Scientific Reporting Skills Assessed using a Rubric focused on Scientific Reporting Skills ± 30 good ; ± 70 need more exercise, need a lot of feedback too much work? impossible? method? Peer Assessment, using the same Rubric, with Peer Feedback, ... Rubrics is not only a method to evaluate these students, but also a skill to be mastered by them. Problem: In Bb Course Tool Self and Peer Assessment : No Rubrics ! ?
  5. 5. Copy/Paste the Rubric “Criteria” and “Levels of Achievement” to spreadsheet: Simulate Rubric & Feedback by a Bb Test
  6. 6. Some basic Hocus-Pocus in the spreadsheet... Simulate Rubric & Feedback by a Bb Test
  7. 7. Copy & Paste this to a notepad document... Simulate Rubric & Feedback by a Bb Test
  8. 8. and upload this notepad document as questions in a Bb-Learn Test: Criteria (29 in this case) Levels of Achievement (MC - questions) Feedback-field (SA - questions) Simulate Rubric & Feedback by a Bb Test
  9. 9. Students Choose Theme > Write > Submit Paper Groupset: Choose one of three themes as subject for your paper... Instructions specific for chosen theme Adaptive Release, specific by theme Only one assignment, common for different themes. Adaptive Release, common for all themes
  10. 10. Random Distribution of Submitted Papers Assignment File Download Theme Group Members export E X C E L Random Number for each submission Sort by Theme and Random Number Replace RN with a “Submission Code”... RENAME COPY TO
  11. 11. Excel parsing filenames randomisation (see one slide up) constructing anonymous filenames constructing batch-commands for renaming and copying Content Collection CC map of students... Blackboard Drive Link Content Collection map to Windows Drive letter Command Window Execute Rename and Copy batch commands Method to automate the delivery of a set of dedicated documents to the respective, individual students. . Other methods possible, but a native Bb-Learn method is missing. Tools for this Distribution of Papers
  12. 12. Student in role of Peer Reviewer Peer Assessment 2 papers > 2 “tests” 29 criteria: level & feedback Finally: Self assessment: 3th test
  13. 13. Redistributing Peer Feedback Paste in the spreadsheet VLookup the author of the reviewed paper Copy to CC map of author using same techniques as for distributing the papers, but in the other direction! here we need a VBA script
  14. 14. Overall Process, Student View Choose theme, Write paper, Submit paper Receive 1st paper to review > peer assess it using rubric & feedback Receive 2d paper to review > peer assess it using rubric & feedback Self assess own paper using rubric & feedback Receive 1st and 2d peer assessment & feedback on own paper Revision and Final Submission of own paper
  15. 15. Conclusion Bb Learn tools for communication Excel & VBA & BbDrive for processing Worth to integrate (partly) in Bb ! complex combination of relative simple tools
  16. 16. End of Part 1 But what if the paper and the Peer Feedback isn’t individual?
  17. 17. Peer Review between groups Hubert Nachtegaele, Educational Technologist Christine Verclairen, Assistant Academic Staff Prof. Dr. ing. Cedric Vuye Ing. Tom Verheyden, Teaching Assistant Faculty of Applied Engineering. Extension to the Bb-Learn Tool “Self and Peer Assessment”
  18. 18. Why?8 group reports (about site visits) Extend to (>) 15 groups 15 groups (4 students) X Extend to (>) 8 site visits How to make students reflect as a group about their writings? each group for each site visit writes: 1 report reviews: 2 reports
  19. 19. How? Prepare Groups and Group Assignments for submission of reports for submission of review of reports Prepare Course Content Collection maps for each group to deliver the reports to review (posted by other groups) to deliver the reviews of their own reports (reviewed by other groups) Prepare a structured Excel Table to randomize the reports for reviewing by other groups to construct batch commands for submitting posted reports to reviewers to construct batch commands for submitting reviews of the reports back to the group who was author of the report Following pictures show more details...
  20. 20. Prepare groups and groupassignments... 1. Design a process for one site visit, independent of the number of groups 2. Repeat that process for each site visit site visit 1 deadline report: xx/xx/xxxx site visit 2 deadline report: xx/xx/xxxx deadline reviews visit 1: xx/xx/xxxx site visit 3 deadline report: xx/xx/xxxx deadline reviews visit 2: xx/xx/xxxx Report of the second activity Reviews of foregoing reports (reports that were written by two other groups)
  21. 21. Distribution of reports to review... Not possible to create subfolders Not possible to automate copying to ... Alternative: Create Course Content Collection map for each group, set permissions based on group membership, batch copying is possible using BbDrive
  22. 22. Download group submissions gradebook_ report1_...... .zip Unzip & list report filenames …and paste them in the prepared Excel table List report filenames…
  23. 23. Copy report to random reviewers group randomize order of groups create batch commands to copy reports to random “next” group Connect to Course Content Collection maps for groups using BbDrive. BbDrive Execute the batch commands to distribute the reports to the group that has to review that report.
  24. 24. Return delivery of the reviews Reviewers download: ToReview1_score=.docx Reviewers do their review work, using the criteria in the scoring form incorporated in the rport template. Reviewers incorporate the overall score in the filename of the review: ToReview1_score=19.docx Reviewers post their review (assignment “Review X”) Course Leader initiates the automated return delivery guided by the Excel.
  25. 25. Conclusion Intention, educational, for these groups: After repeating this report/review process for 8 site visits, the groups write clear and correct reports. Intention, educational, for the institution: A new Learning Tool for enhancing output of teamwork. What can simplify this method? Accessibility of the “file exchange” group tool through BbDrive ⇒ no need to create CC maps for the groups
  26. 26. End of Part 2 And the assessment within the group?
  27. 27. Hubert Nachtegaele, Educational Technologist Raoul Deuss, Domain Coördinator Ecampus Peer Assessment within Groups Group Members evaluate their Teamwork Skills
  28. 28. Construction of the PA-test From the Groups Membership in Blackboard and the list of Teamwork Skills ...
  29. 29. Construction of the PA-test The Course Leader starts a VBA-script to create a Bb-test...
  30. 30. The result is a Bb-test in which students assess these Teamwork Skills of all the Group Members, self-assessment included. The PA-test
  31. 31. Meaning of the Scores By means of the scores in the PA-test you evaluate yourself and the other members of your group: You assign scores from 1 to 5. The meaning of these figures is: 5 = For this criterium, the contribution of group member ... was better than the average of the group 4 = For this criterium, the contribution of group member ... was roughly the average of the group 3 = For this criterium, the contribution of group member ... was less than the average of the group 2 = For this criterium, group member ... contributed nothing at all 1 = For this criterium, group member ... was a nuisance to the group Assign a score for all criteria for all members of your group (yourself included).
  32. 32. PA-rapports After the download of the test results, another VBA-script calculates scores and creates individual reports. These reports are sent automatically to the personal CC map of each student. Only the course leader sees all the details.
  33. 33. End of Part 3
  34. 34. Peer Feedback via My Grades the most appropriate way? Hubert Nachtegaele, Educational Technologist
  35. 35. Upload Comments to the Grade Center In the Bb documentation we found that it is possible to download and upload not only the grades, but also the “Comments”. We decided to use this to return Peer Feedback to the students.
  36. 36. Feedback experiment We constructed a test consisting of 3 questions about the oral presentation of each of the 24 students ⇒ 3 x 24 questions After the students gave feedback about each other via this test, we downloaded the results and ... Using this “test”, each student can give feedback to any other student.
  37. 37. Feedback from Excel to Grade Center After the students gave feedback on each others presentation via this test, we downloaded the results, reordered and combined these pieces of feedback in function of the assessed student In CSV-form, this is uploaded to the Grade Center column...
  38. 38. Peer Feedback: Student View The result of this procedure is that the students see their peer feedback the same way as they see the feedback from instructors: through “My Grades”.
  39. 39. End of Part 4
  40. 40. Overall Conclusion These peer (review, feedback, assessment) methods give satisfactory educational results Managing these processes can be simplified by integrating some elements in future releases of Bb Learn (Ultra)... ● make the use of rubrics also available, in some way, to students for peer assessment ● simple tool for submitting dedicated documents to respective students, eg via My Grades ● limit focus of Bb Learn tool “Self & Peer Assessment” to “group(s) of students” instead of “all students”
  41. 41. End

×