Presenter: Hubert Nachtegaele Organisation: Universiteit Antwerpen Description: Our teaching staff asks support for types of “Self and Peer Assessment” which are not possible with the S&PA tool incorporated in Bb Learn. Our Blackboard Support Team tries to support the desired types of S&PA by creative combinations solely of Bb Learn Course Tools, without using external tools (except Excel). In this session we will show how we realize this for different types of S&PA: “Groups reviewing assignments of other groups”, “Students assessing their peers and themselves within a group”, “Peer Reviewing using a rubric”, and "Anonymous Peer Feedback for oral presentations"

1. Peer Feedback methods
based on Bb Learn Tools
Extensions to the Bb-Learn Tool
“Self and Peer Assessment”

2. Peer Feedback Methods
based on Blackboard Learn Tools
Peer Review with Rubrics
Peer Review between Groups
Peer Assessment within Groups
Peer Feedback via My Grades
Presentation by:
Hubert Nachtegaele, Educational Technologist
Christine Verclairen, Assistant Academic Staff
Extensions to the Bb-Learn Tool
“Self and Peer Assessment”

3. Peer Review
with Rubrics
Cartoon by Nick D Kim, strange-matter.net (please see site for terms of reuse)
Extensions to the Bb-Learn Tool
“Self and Peer Assessment”
Hubert Nachtegaele, Educational Technologist
Annick De Vylder, Assistant Academic Staff
Prof. Dr. Jelle Boeve-de Pauw and prof. Dr. Piet
Van den Bossche, Faculty of Social Sciences

4. Why?
± 100 students “Master of Instructional and Educational Sciences”
Assignment: write paper, apply/show Scientific Reporting Skills
Assessed using a Rubric focused on Scientific Reporting Skills
± 30 good ; ± 70 need more exercise, need a lot of feedback
too much work?
impossible?
method?
Peer Assessment, using the same Rubric,
with Peer Feedback, ...
Rubrics is not only
a method to
evaluate these
students, but also
a skill to be
mastered by them.
Problem:
In Bb Course Tool Self and Peer Assessment : No Rubrics !
?

5. Copy/Paste the Rubric “Criteria” and “Levels of Achievement” to spreadsheet:
Simulate Rubric & Feedback by a Bb Test

6. Some basic Hocus-Pocus in the spreadsheet...
Simulate Rubric & Feedback by a Bb Test

7. Copy & Paste this to a notepad document...
Simulate Rubric & Feedback by a Bb Test

8. and upload this notepad document as questions in a Bb-Learn Test:
Criteria
(29 in this case)
Levels of Achievement
(MC - questions)
Feedback-field
(SA - questions)
Simulate Rubric & Feedback by a Bb Test

9. Students Choose Theme > Write > Submit Paper
Groupset:
Choose one of three themes
as subject for your paper...
Instructions specific
for chosen theme
Adaptive Release,
specific by theme
Only one assignment,
common for different themes.
Adaptive Release,
common for all themes

10. Random Distribution of Submitted Papers
Assignment File Download
Theme Group Members export
E
X
C
E
L
Random Number for each submission
Sort by Theme and Random Number
Replace RN with a “Submission Code”...
RENAME
COPY TO

11. Excel
parsing filenames
randomisation (see one slide up)
constructing anonymous filenames
constructing batch-commands for renaming and copying
Content Collection
CC map of students...
Blackboard Drive
Link Content Collection map to Windows Drive letter
Command Window
Execute Rename and Copy batch commands
Method to automate the
delivery of a set of
dedicated documents to
the respective, individual
students.
.
Other methods possible,
but a native Bb-Learn
method is missing.
Tools for this Distribution of Papers

12. Student in role of
Peer Reviewer
Peer Assessment
2 papers > 2 “tests”
29 criteria: level & feedback
Finally:
Self assessment: 3th test

13. Redistributing Peer Feedback
Paste in the
spreadsheet
VLookup the author of
the reviewed paper
Copy to CC map of
author using same
techniques as for
distributing the papers,
but in the other direction!
here we need
a VBA script

14. Overall Process, Student View
Choose theme, Write paper, Submit paper
Receive 1st paper to review > peer assess it using rubric & feedback
Receive 2d paper to review > peer assess it using rubric & feedback
Self assess own paper using rubric & feedback
Receive 1st and 2d peer assessment & feedback on own paper
Revision and Final Submission of own paper

15. Conclusion
Bb Learn tools for communication
Excel & VBA & BbDrive for processing
Worth to integrate (partly) in Bb !
complex combination of
relative simple tools

16. End of Part 1
But what if the paper and the Peer Feedback isn’t individual?

17. Peer Review
between groups
Hubert Nachtegaele, Educational Technologist
Christine Verclairen, Assistant Academic Staff
Prof. Dr. ing. Cedric Vuye
Ing. Tom Verheyden, Teaching Assistant
Faculty of Applied Engineering.
Extension to the Bb-Learn Tool
“Self and Peer Assessment”

18. Why?8 group reports
(about site visits)
Extend to
(>) 15 groups
15 groups
(4 students) X
Extend to
(>) 8 site visits
How to make students reflect as a group
about their writings?
each group
for each
site visit
writes:
1 report
reviews:
2 reports

19. How?
Prepare Groups and Group Assignments
for submission of reports
for submission of review of reports
Prepare Course Content Collection maps for each group
to deliver the reports to review (posted by other groups)
to deliver the reviews of their own reports (reviewed by other groups)
Prepare a structured Excel Table
to randomize the reports for reviewing by other groups
to construct batch commands for submitting posted reports to reviewers
to construct batch commands for submitting reviews of the reports back to
the group who was author of the report
Following pictures show more details...

20. Prepare groups and groupassignments...
1. Design a process for one
site visit, independent of
the number of groups
2. Repeat that process
for each site visit
site visit 1
deadline report: xx/xx/xxxx
site visit 2
deadline report: xx/xx/xxxx
deadline reviews visit 1:
xx/xx/xxxx
site visit 3
deadline report: xx/xx/xxxx
deadline reviews visit 2:
xx/xx/xxxx
Report of the
second activity
Reviews of
foregoing reports
(reports that were
written by two other
groups)

21. Distribution of reports to review...
Not possible to create
subfolders
Not possible to automate
copying to ...
Alternative:
Create Course Content
Collection map for each
group,
set permissions based on
group membership,
batch copying is possible
using BbDrive

22. Download group submissions
gradebook_
report1_......
.zip
Unzip & list
report
filenames
…and paste them in the
prepared Excel table List report filenames…

23. Copy report to random reviewers group
randomize order
of groups
create batch commands to copy
reports to random “next” group
Connect to Course Content
Collection maps for groups
using BbDrive.
BbDrive
Execute the batch commands to
distribute the reports to the group that
has to review that report.

24. Return delivery of the reviews
Reviewers download:
ToReview1_score=.docx
Reviewers do their review work, using
the criteria in the scoring form
incorporated in the rport template.
Reviewers incorporate the overall
score in the filename of the review:
ToReview1_score=19.docx
Reviewers post their review
(assignment “Review X”)
Course Leader initiates the automated
return delivery guided by the Excel.

25. Conclusion
Intention, educational, for these groups:
After repeating this report/review process for 8 site visits,
the groups write clear and correct reports.
Intention, educational, for the institution:
A new Learning Tool for enhancing output of teamwork.
What can simplify this method?
Accessibility of the “file exchange” group tool through
BbDrive ⇒ no need to create CC maps for the groups

26. End of Part 2
And the assessment within the group?

27. Hubert Nachtegaele, Educational Technologist
Raoul Deuss, Domain Coördinator Ecampus
Peer Assessment
within Groups
Group Members evaluate
their Teamwork Skills

28. Construction of the PA-test
From the Groups Membership in Blackboard and the list of Teamwork Skills ...

29. Construction of the PA-test
The Course Leader starts a VBA-script to create a Bb-test...

30. The result is a Bb-test in which students assess these
Teamwork Skills of all the Group Members, self-assessment included.
The PA-test

31. Meaning of the Scores
By means of the scores in the PA-test you evaluate yourself and the other members of your group:
You assign scores from 1 to 5. The meaning of these figures is:
5 = For this criterium, the contribution of group member ... was better than the average of the group
4 = For this criterium, the contribution of group member ... was roughly the average of the group
3 = For this criterium, the contribution of group member ... was less than the average of the group
2 = For this criterium, group member ... contributed nothing at all
1 = For this criterium, group member ... was a nuisance to the group
Assign a score for all criteria for all members of your group (yourself included).

32. PA-rapports
After the download of the test results,
another VBA-script calculates scores and
creates individual reports.
These reports are sent automatically to the
personal CC map of each student.
Only the course leader sees all the details.

33. End of Part 3

34. Peer Feedback
via My Grades
the most appropriate way?
Hubert Nachtegaele, Educational Technologist

35. Upload Comments to the Grade Center
In the Bb documentation we found
that it is possible to download and
upload not only the grades, but also
the “Comments”.
We decided to use this to return
Peer Feedback to the students.

36. Feedback experiment
We constructed a test consisting of 3 questions about the oral
presentation of each of the 24 students ⇒ 3 x 24 questions
After the students gave feedback about each other via this
test, we downloaded the results and ...
Using this “test”, each student can
give feedback to any other student.

37. Feedback from Excel to Grade Center
After the students gave feedback on each others presentation via
this test, we downloaded the results, reordered and combined
these pieces of feedback in function of the assessed student
In CSV-form, this is uploaded to
the Grade Center column...

38. Peer Feedback: Student View
The result of this
procedure is that the
students see their peer
feedback the same way as
they see the feedback
from instructors:
through “My Grades”.

39. End of Part 4

40. Overall Conclusion
These peer (review, feedback, assessment) methods
give satisfactory educational results
Managing these processes can be simplified by integrating
some elements in future releases of Bb Learn (Ultra)...
● make the use of rubrics also available, in some way, to
students for peer assessment
● simple tool for submitting dedicated documents to
respective students, eg via My Grades
● limit focus of Bb Learn tool “Self & Peer Assessment” to
“group(s) of students” instead of “all students”

41. End