Peer feedback wordt binnen het hoger onderwijs steeds vaker ingezet. Helaas ontbreekt soms de gewenste diepgang. Het doel van dit project is om peer feedback op innovatieve wijze in te richten en te beoordelen. FeedbackFruits en Wageningen University hebben daartoe een tool ontwikkeld voor fair participation grading: een eerlijke beoordeling van peer feedback. Hiermee kunnen studenten zelf hun beste bijdrages verzamelen, die vervolgens worden beoordeeld door de docent. Fair participation grading is via experimenteel onderzoek geëvalueerd. Het didactisch concept en de eerste resultaten presenteren we tijdens de SURF Onderwijsdagen.

1. Best Contribution Grading (BCG)
M.C. (Cora) Busstra, PhD – Wageningen University
SURF OWD2019, 5-6 nov 2019

2. Introduction
Peer review by students: in theory
 Stimulates problem solving skills
 Better understanding by reflecting on others & self
Practice
− Lack of critical depth
− If not graded: Participation is low
Solution
Best contribution grading (BCG)
2
This presentation
 The idea of BCG
 How it works in
practice
 Preliminary evaluation
results

3. Pro’s of ‘Best Contribution grading’
3

4. ‘Best contribution grading’ (in practice)
4
Write document
Make video etc
Give peer review

5. ‘Best contribution grading’ (BCG)
5

6. ‘Best contribution grading’ (in practice)
6
Write document
Make video
Give peer review

7. ‘Best contribution grading’ (BCG)
7

8. ‘Best contribution grading’ (in practice)
8
Write document
Make video
Give peer review

9. ‘Best contribution grading’ (BCG)
9

10. ‘Best contribution grading’ (in practice)
10
Write document
Make video
Give peer review

11. Evaluation of BCG: cross-over experiments
11
Three experiments
 Presentation skills (MSc) (n=93)
 Proposal writing
 Engineering (n=170)
Peer review
Assignment 1 Assignment 2
 Participation grade (100%)
 Review characteristics
 Review quality
 Students’ satisfaction
 Teachers’ satisfaction

12. Review characteristics
12
 Sentiment analyses: no difference
 Depth/correctness of review  Not yet analysed
0
20
40
60
80
100
120
140
160
180
Exp 1 Exp 2 Exp 3
Time on Review (min)
0
50
100
150
200
250
300
350
400
Exp 1 Exp 2 Exp 3
Words in Review
BCG
No BCG

13. Results: students perception of BCG
13
5-point scale (disagree-agree)
Exp 1
(n=70)
Exp 2
(n=98)
Exp 3
(n=57)
BCG motivated to participate 3.5 4.0 3.4
BCG motivated to provide high quality feedback 3.8 4.1 3.6
Rubric was useful 3.9 3.5 3.6
Able to select best contributions 3.8 4.2 3.8
Fair that only best comments were graded 3.5 3.7 2.9
Recommend BCG in other courses 3.5 3.8 2.9

14. Results: effect of BCG on perception of peer review
14
Exp 1 Exp 2 Exp 3
Yes
(n=45)
No
(n=25)
Yes
(n=65)
No
(n=33)
Yes
(n=34)
No
(n=23)
Insightful to see work of others
4.1 4.2 4.2 4.3 3.9 3.6
Giving feedback improved own
understanding 4.1* 3.6 * 3.9 4.1 3.6 3.4
Quality of feedback I provided was
satisfactory 4.1 4.0 4.1 3.9 3.7 3.3
Enjoyed giving feedback
3.5 3.4 3.8 3.7 2.7 2.5
Quality of received feedback was
satisfactory
3.7 3.8 4.0 3.9 3.3 3.0

15. Teachers perception
15
“Students do not
complain about
the peer feedback
anymore”
“I have the feeling
the quality of the
peer feedback is
much higher this
year”

16.  Evaluation: Lack of statistical power
● Upscaling to more students and institutions
● Low response on evaluation questionnaire
 Promising results
 Does it improve quality according to judgement teacher
(rubric)?
 Does it work for online discussions?
 For which target groups/content domain?
Conclusion
16

17. Questions?
17