Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Peer Feedback methods
based on Bb Learn Tools
Extensions to the Bb-Learn Tool
“Self and Peer Assessment”
Peer Feedback Methods
based on Blackboard Learn Tools
Peer Review with Rubrics
Peer Review between Groups
Peer Assessment ...
Peer Review
with Rubrics
Cartoon by Nick D Kim, strange-matter.net (please see site for terms of reuse)
Extensions to the ...
Why?
± 100 students “Master of Instructional and Educational Sciences”
Assignment: write paper, apply/show Scientific Repo...
Copy/Paste the Rubric “Criteria” and “Levels of Achievement” to spreadsheet:
Simulate Rubric & Feedback by a Bb Test
Some basic Hocus-Pocus in the spreadsheet...
Simulate Rubric & Feedback by a Bb Test
Copy & Paste this to a notepad document...
Simulate Rubric & Feedback by a Bb Test
and upload this notepad document as questions in a Bb-Learn Test:
Criteria
(29 in this case)
Levels of Achievement
(MC - q...
Students Choose Theme > Write > Submit Paper
Groupset:
Choose one of three themes
as subject for your paper...
Instruction...
Random Distribution of Submitted Papers
Assignment File Download
Theme Group Members export
E
X
C
E
L
Random Number for ea...
Excel
parsing filenames
randomisation (see one slide up)
constructing anonymous filenames
constructing batch-commands for ...
Student in role of
Peer Reviewer
Peer Assessment
2 papers > 2 “tests”
29 criteria: level & feedback
Finally:
Self assessme...
Redistributing Peer Feedback
Paste in the
spreadsheet
VLookup the author of
the reviewed paper
Copy to CC map of
author us...
Overall Process, Student View
Choose theme, Write paper, Submit paper
Receive 1st paper to review > peer assess it using r...
Conclusion
Bb Learn tools for communication
Excel & VBA & BbDrive for processing
Worth to integrate (partly) in Bb !
compl...
End of Part 1
But what if the paper and the Peer Feedback isn’t individual?
Peer Review
between groups
Hubert Nachtegaele, Educational Technologist
Christine Verclairen, Assistant Academic Staff
Pro...
Why?8 group reports
(about site visits)
Extend to
(>) 15 groups
15 groups
(4 students) X
Extend to
(>) 8 site visits
How t...
How?
Prepare Groups and Group Assignments
for submission of reports
for submission of review of reports
Prepare Course Con...
Prepare groups and groupassignments...
1. Design a process for one
site visit, independent of
the number of groups
2. Repe...
Distribution of reports to review...
Not possible to create
subfolders
Not possible to automate
copying to ...
Alternative...
Download group submissions
gradebook_
report1_......
.zip
Unzip & list
report
filenames
…and paste them in the
prepared Ex...
Copy report to random reviewers group
randomize order
of groups
create batch commands to copy
reports to random “next” gro...
Return delivery of the reviews
Reviewers download:
ToReview1_score=.docx
Reviewers do their review work, using
the criteri...
Conclusion
Intention, educational, for these groups:
After repeating this report/review process for 8 site visits,
the gro...
End of Part 2
And the assessment within the group?
Hubert Nachtegaele, Educational Technologist
Raoul Deuss, Domain Coördinator Ecampus
Peer Assessment
within Groups
Group M...
Construction of the PA-test
From the Groups Membership in Blackboard and the list of Teamwork Skills ...
Construction of the PA-test
The Course Leader starts a VBA-script to create a Bb-test...
The result is a Bb-test in which students assess these
Teamwork Skills of all the Group Members, self-assessment included....
Meaning of the Scores
By means of the scores in the PA-test you evaluate yourself and the other members of your group:
You...
PA-rapports
After the download of the test results,
another VBA-script calculates scores and
creates individual reports.
T...
End of Part 3
Peer Feedback
via My Grades
the most appropriate way?
Hubert Nachtegaele, Educational Technologist
Upload Comments to the Grade Center
In the Bb documentation we found
that it is possible to download and
upload not only t...
Feedback experiment
We constructed a test consisting of 3 questions about the oral
presentation of each of the 24 students...
Feedback from Excel to Grade Center
After the students gave feedback on each others presentation via
this test, we downloa...
Peer Feedback: Student View
The result of this
procedure is that the
students see their peer
feedback the same way as
they...
End of Part 4
Overall Conclusion
These peer (review, feedback, assessment) methods
give satisfactory educational results
Managing these ...
End
Upcoming SlideShare
Loading in …5
×

TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

649 views

Published on

Presenter: Hubert Nachtegaele
Organisation: Universiteit Antwerpen

Description: Our teaching staff asks support for types of “Self and Peer Assessment” which are not possible with the S&PA tool incorporated in Bb Learn.

Our Blackboard Support Team tries to support the desired types of S&PA by creative combinations solely of Bb Learn Course Tools, without using external tools (except Excel). In this session we will show how we realize this for different types of S&PA: “Groups reviewing assignments of other groups”, “Students assessing their peers and themselves within a group”, “Peer Reviewing using a rubric”, and "Anonymous Peer Feedback for oral presentations"

Published in: Education
  • Be the first to comment

  • Be the first to like this

TLC2016 - Peer Review, Peer Assessment, and Peer Feedback methods based on Blackboard Learn Tools

  1. 1. Peer Feedback methods based on Bb Learn Tools Extensions to the Bb-Learn Tool “Self and Peer Assessment”
  2. 2. Peer Feedback Methods based on Blackboard Learn Tools Peer Review with Rubrics Peer Review between Groups Peer Assessment within Groups Peer Feedback via My Grades Presentation by: Hubert Nachtegaele, Educational Technologist Christine Verclairen, Assistant Academic Staff Extensions to the Bb-Learn Tool “Self and Peer Assessment”
  3. 3. Peer Review with Rubrics Cartoon by Nick D Kim, strange-matter.net (please see site for terms of reuse) Extensions to the Bb-Learn Tool “Self and Peer Assessment” Hubert Nachtegaele, Educational Technologist Annick De Vylder, Assistant Academic Staff Prof. Dr. Jelle Boeve-de Pauw and prof. Dr. Piet Van den Bossche, Faculty of Social Sciences
  4. 4. Why? ± 100 students “Master of Instructional and Educational Sciences” Assignment: write paper, apply/show Scientific Reporting Skills Assessed using a Rubric focused on Scientific Reporting Skills ± 30 good ; ± 70 need more exercise, need a lot of feedback too much work? impossible? method? Peer Assessment, using the same Rubric, with Peer Feedback, ... Rubrics is not only a method to evaluate these students, but also a skill to be mastered by them. Problem: In Bb Course Tool Self and Peer Assessment : No Rubrics ! ?
  5. 5. Copy/Paste the Rubric “Criteria” and “Levels of Achievement” to spreadsheet: Simulate Rubric & Feedback by a Bb Test
  6. 6. Some basic Hocus-Pocus in the spreadsheet... Simulate Rubric & Feedback by a Bb Test
  7. 7. Copy & Paste this to a notepad document... Simulate Rubric & Feedback by a Bb Test
  8. 8. and upload this notepad document as questions in a Bb-Learn Test: Criteria (29 in this case) Levels of Achievement (MC - questions) Feedback-field (SA - questions) Simulate Rubric & Feedback by a Bb Test
  9. 9. Students Choose Theme > Write > Submit Paper Groupset: Choose one of three themes as subject for your paper... Instructions specific for chosen theme Adaptive Release, specific by theme Only one assignment, common for different themes. Adaptive Release, common for all themes
  10. 10. Random Distribution of Submitted Papers Assignment File Download Theme Group Members export E X C E L Random Number for each submission Sort by Theme and Random Number Replace RN with a “Submission Code”... RENAME COPY TO
  11. 11. Excel parsing filenames randomisation (see one slide up) constructing anonymous filenames constructing batch-commands for renaming and copying Content Collection CC map of students... Blackboard Drive Link Content Collection map to Windows Drive letter Command Window Execute Rename and Copy batch commands Method to automate the delivery of a set of dedicated documents to the respective, individual students. . Other methods possible, but a native Bb-Learn method is missing. Tools for this Distribution of Papers
  12. 12. Student in role of Peer Reviewer Peer Assessment 2 papers > 2 “tests” 29 criteria: level & feedback Finally: Self assessment: 3th test
  13. 13. Redistributing Peer Feedback Paste in the spreadsheet VLookup the author of the reviewed paper Copy to CC map of author using same techniques as for distributing the papers, but in the other direction! here we need a VBA script
  14. 14. Overall Process, Student View Choose theme, Write paper, Submit paper Receive 1st paper to review > peer assess it using rubric & feedback Receive 2d paper to review > peer assess it using rubric & feedback Self assess own paper using rubric & feedback Receive 1st and 2d peer assessment & feedback on own paper Revision and Final Submission of own paper
  15. 15. Conclusion Bb Learn tools for communication Excel & VBA & BbDrive for processing Worth to integrate (partly) in Bb ! complex combination of relative simple tools
  16. 16. End of Part 1 But what if the paper and the Peer Feedback isn’t individual?
  17. 17. Peer Review between groups Hubert Nachtegaele, Educational Technologist Christine Verclairen, Assistant Academic Staff Prof. Dr. ing. Cedric Vuye Ing. Tom Verheyden, Teaching Assistant Faculty of Applied Engineering. Extension to the Bb-Learn Tool “Self and Peer Assessment”
  18. 18. Why?8 group reports (about site visits) Extend to (>) 15 groups 15 groups (4 students) X Extend to (>) 8 site visits How to make students reflect as a group about their writings? each group for each site visit writes: 1 report reviews: 2 reports
  19. 19. How? Prepare Groups and Group Assignments for submission of reports for submission of review of reports Prepare Course Content Collection maps for each group to deliver the reports to review (posted by other groups) to deliver the reviews of their own reports (reviewed by other groups) Prepare a structured Excel Table to randomize the reports for reviewing by other groups to construct batch commands for submitting posted reports to reviewers to construct batch commands for submitting reviews of the reports back to the group who was author of the report Following pictures show more details...
  20. 20. Prepare groups and groupassignments... 1. Design a process for one site visit, independent of the number of groups 2. Repeat that process for each site visit site visit 1 deadline report: xx/xx/xxxx site visit 2 deadline report: xx/xx/xxxx deadline reviews visit 1: xx/xx/xxxx site visit 3 deadline report: xx/xx/xxxx deadline reviews visit 2: xx/xx/xxxx Report of the second activity Reviews of foregoing reports (reports that were written by two other groups)
  21. 21. Distribution of reports to review... Not possible to create subfolders Not possible to automate copying to ... Alternative: Create Course Content Collection map for each group, set permissions based on group membership, batch copying is possible using BbDrive
  22. 22. Download group submissions gradebook_ report1_...... .zip Unzip & list report filenames …and paste them in the prepared Excel table List report filenames…
  23. 23. Copy report to random reviewers group randomize order of groups create batch commands to copy reports to random “next” group Connect to Course Content Collection maps for groups using BbDrive. BbDrive Execute the batch commands to distribute the reports to the group that has to review that report.
  24. 24. Return delivery of the reviews Reviewers download: ToReview1_score=.docx Reviewers do their review work, using the criteria in the scoring form incorporated in the rport template. Reviewers incorporate the overall score in the filename of the review: ToReview1_score=19.docx Reviewers post their review (assignment “Review X”) Course Leader initiates the automated return delivery guided by the Excel.
  25. 25. Conclusion Intention, educational, for these groups: After repeating this report/review process for 8 site visits, the groups write clear and correct reports. Intention, educational, for the institution: A new Learning Tool for enhancing output of teamwork. What can simplify this method? Accessibility of the “file exchange” group tool through BbDrive ⇒ no need to create CC maps for the groups
  26. 26. End of Part 2 And the assessment within the group?
  27. 27. Hubert Nachtegaele, Educational Technologist Raoul Deuss, Domain Coördinator Ecampus Peer Assessment within Groups Group Members evaluate their Teamwork Skills
  28. 28. Construction of the PA-test From the Groups Membership in Blackboard and the list of Teamwork Skills ...
  29. 29. Construction of the PA-test The Course Leader starts a VBA-script to create a Bb-test...
  30. 30. The result is a Bb-test in which students assess these Teamwork Skills of all the Group Members, self-assessment included. The PA-test
  31. 31. Meaning of the Scores By means of the scores in the PA-test you evaluate yourself and the other members of your group: You assign scores from 1 to 5. The meaning of these figures is: 5 = For this criterium, the contribution of group member ... was better than the average of the group 4 = For this criterium, the contribution of group member ... was roughly the average of the group 3 = For this criterium, the contribution of group member ... was less than the average of the group 2 = For this criterium, group member ... contributed nothing at all 1 = For this criterium, group member ... was a nuisance to the group Assign a score for all criteria for all members of your group (yourself included).
  32. 32. PA-rapports After the download of the test results, another VBA-script calculates scores and creates individual reports. These reports are sent automatically to the personal CC map of each student. Only the course leader sees all the details.
  33. 33. End of Part 3
  34. 34. Peer Feedback via My Grades the most appropriate way? Hubert Nachtegaele, Educational Technologist
  35. 35. Upload Comments to the Grade Center In the Bb documentation we found that it is possible to download and upload not only the grades, but also the “Comments”. We decided to use this to return Peer Feedback to the students.
  36. 36. Feedback experiment We constructed a test consisting of 3 questions about the oral presentation of each of the 24 students ⇒ 3 x 24 questions After the students gave feedback about each other via this test, we downloaded the results and ... Using this “test”, each student can give feedback to any other student.
  37. 37. Feedback from Excel to Grade Center After the students gave feedback on each others presentation via this test, we downloaded the results, reordered and combined these pieces of feedback in function of the assessed student In CSV-form, this is uploaded to the Grade Center column...
  38. 38. Peer Feedback: Student View The result of this procedure is that the students see their peer feedback the same way as they see the feedback from instructors: through “My Grades”.
  39. 39. End of Part 4
  40. 40. Overall Conclusion These peer (review, feedback, assessment) methods give satisfactory educational results Managing these processes can be simplified by integrating some elements in future releases of Bb Learn (Ultra)... ● make the use of rubrics also available, in some way, to students for peer assessment ● simple tool for submitting dedicated documents to respective students, eg via My Grades ● limit focus of Bb Learn tool “Self & Peer Assessment” to “group(s) of students” instead of “all students”
  41. 41. End

×