This document summarizes a paper that analyzes self and peer assessment (SPA) as a strategic game. It defines SPA and models it as a game where students select ratings to maximize their marks. The analysis predicts a Nash equilibrium where all students rate themselves and peers equally to receive equal marks. A case study of SPA contradicted this, as students generally rated each other equally. The document discusses reasons for this, including reciprocity bias and factors outside the SPA design impacting student behavior. It concludes the meaning of SPA ratings is unclear and a cooperative game theory approach may improve assessment design.

1. Reliability of Self and Peer Assessment of Group
Work in Higher Education
Mehdi Chowdhury
Department of Accounting, Finance and Economics
Bournemouth University
SLTCC, Southampton, 24.06.2016

2. Table of Contents
Background and Summary
Literature review
SPA as a strategic form game
A Case Study
Discussion and Conclusion

3. What is Self and peer assessment?
In group work based assessments, contributions of individual
members are not visible from the final product.
Group members rate contributions of each other and report to
the assessor.
The reported ratings are utilised to assign individual marks by
the assessor.

4. An example of SPA
Self assessment:
Much more than all other group members: 25
More than other group members: 20
Same as other group members: 15
Less than other group members: 10
Much less than all other group members: 5
Peer assessment:
X contributed much more than all other group members: 25
X contributed more than other group members: 20
X contributed the same as other group members: 15
X contributed less than other group members: 10
X contributed much less than all other group members: 5

5. Contribution of the paper
Demonstrates that SPA by structure a strategic form game
and utilises a game theoretic analysis to identify potential
outcomes.
It implies a unique Nash equilibrium where students try to
maximise self mark.
Analysis of a case study : predicts the outcome of that design.
Empirical results contradict with the prediction.
The meaning of ratings are unknown.

6. Table of Contents
Background and Summary
Literature review
SPA as a strategic form game
A Case Study
Discussion and Conclusion

7. Literature review
A number of comprehensive reviews, Falchikov (2005),
Topping (2003, 2009), Dochy et al. (1999).
Spatar et al. (2015) comparison of various methods used for
mapping.
A general tendency to favour SPA.
Burke (1969), students failed to assess objectively, realistically
and tendency of over-grading.
Hanrahan and Isaacs (2001), difficult to judge own work and
discomfort in critiquing and marking others down. Magin
(2001) reciprocity bias.
A number of mapping methods.
No application of game theoretic reasoning.

8. Table of Contents
Background and Summary
Literature review
SPA as a strategic form game
A Case Study
Discussion and Conclusion

9. Definition of a Strategic Form Game
A game in a strategic form (or in normal form) is an ordered
triple G = (N, S, U), such that:
N = {1, 2, ......N} is the finite set of players.
Si is the set of strategies of player i, for every player i ∈ N.
The set of all vector of strategies is
S = S1 × S2 × S2 × ........ × Sn.
ui : S → R is a function associating each vector of strategies
s = (si )i∈N with the pay off ui (s) to player i, for every player
i ∈ N.

10. Nash Equilibrium and Dominant Strategy
A strategy si is a profitable deviation for a player i if
ui (si , s−i ) > ui (s).
A Nash equilibrium is a strategy profile (s∗ = s∗
1 , s∗
2 , ...., s∗
n )
at which no player has a profitable deviation.
A Dominant strategy is a strategy s∗
i for player i that is a
best response to all strategy profiles of other players.

11. SPA as a strategic form game
SPA is a strategic form game as:
Players =⇒ Group members
Strategies =⇒ Reported Self and peer assessment
Utilities =⇒ Individual marks/Satisfaction

12. Structure and assumptions of SPA game
Assume:
I. There are n number of students in a group.
II. The reported self assessment of a student (i = 1, 2, 3, ...n) is
xii ∈ [0, x].
III. The reported peer assessment of a student i about j(i = j) is
xij ∈ [0, x].
IV. The actual self and peer assessment are vii ∈ [0, x] and
vij ∈ [0, x].

13. Structure and assumptions of SPA game (Cont.)
Assume:
Students are capable of assessing self and peer contributions
appropriately.
The assessor expects students to set xii = vii and xij = vij , i.e.
to report truthfully.
The reports are not disclosed by the assessor to group
members.
The utility of a student is increasing in own mark.
Students are indifferent about marks of peers.

14. Dominant strategy (Self Assessment only)
Assume:
V. The award scheme φi is a function φi : xii → R+ . Assume φi
is increasing in xii .
(V)implies, as φi is increasing in xii , the dominant strategy of a
student is to set xii = x as the utility/satisfaction is increasing in
final mark.

15. Dominant strategy(Self and Peer Assessment)
Assume:
VI. A award scheme gi : Rn
+ → R+ and ψi : gi → R+. Assume gi
is increasing in xii and ψi is increasing in gi .
(VI) implies that the assessor uses the reports of SPA of the group
to create a representative number gi . The representative number is
then used to award the mark ψi to the student i. The dominant
strategy of i is to report xii = x for any xji , i.e. the peer
assessment by the other members of the group.

16. Table of Contents
Background and Summary
Literature review
SPA as a strategic form game
A Case Study
Discussion and Conclusion

17. Instructions
A 3000 word group essay consisting of 4 members.
Submitted confidential self and peer assessment emailing to
the tutor.
Tutor instructed that the individual marks will be adjusted
upward or downward based on SPA. Actual marking scheme is
not available in the assignment instruction.
In actual marking, the tutor used a ratio based scheme.
Assignment instruction stated the possibility of face to face
interview to justify rating.

18. Instructions (Cont.)
Self assessment:
Much more than all other group members: 25
More than other group members: 20
Same as other group members: 15
Less than other group members: 10
Much less than all other group members: 5
Peer assessment:
X contributed much more than all other group members: 25
X contributed more than other group members: 20
X contributed the same as other group members: 15
X contributed less than other group members: 10
X contributed much less than all other group members: 5

19. Strategy profiles
Each student selects self and peer ratings from the set of
numbers 5, 10, 15, 20 and 25.
Sm = {sm1, sm2, ......, sm625} are strategies, where
m = 1, 2, 3, 4 are the members and i = 1, 2, ...., 625 are index
of the strategies.
smi is a vector with 4 elements, i.e.
smi = { ai
m1 ai
m2 ai
m3 ai
m4}

20. Strategy profiles (Cont.)
Final outcome looks like,
A =




a11 a12 a13 a14
a21 a22 a23 a24
a31 a32 a33 a34
a41 a42 a43 a44





21. Nash equilibrium
If students play dominant strategies, the outcome is,
A =




25 5 5 5
5 25 5 5
5 5 25 5
5 5 5 25





22. Nash equilibrium (cont.)
The ratio of the contribution of a student is:
4
m=1
am1
4
m=1
4
m=1
amm
=
40
160
= 0.25
Therefore if students play their dominant strategies, the ratio of
contribution on a student is 25% implying that the grade of a
student is the same as the other group members. Nevertheless, it
is the only Nash equilibrium as any deviation decreases the grade
of a student.

23. Data analysis
48 students in 12 groups
Figure : Distribution of Self Assessment

24. Data analysis (Cont.)
Figure : Distribution of Average Peer Assessment

25. Data analysis (Cont.)
Table : Summary Statistics
Obs. Mean Standard Dev. Min Max
Self Assessment 48 15.94 3.67 5 25
Peer Assessment 48 15.35 1.91 11.67 25

26. Data analysis (Cont.)
Figure : Assessment of the self compared to the peers

27. Actual rating matrix
The matrix of actual rating is,
A =




15 15 15 15
15 15 15 15
15 15 15 15
15 15 15 15





28. Table of Contents
Background and Summary
Literature review
SPA as a strategic form game
A Case Study
Discussion and Conclusion

29. Discussion and Conclusion
Students have not adopted dominant strategies. In general
they reported all equal.
Reciprocity bias?
Altruism and Guilt Aversion?
A trust game? The tutor trusts students, and students
reciprocate?
Many factors are outside of the design of SPA.
Spatar et al. (2015) method only uses peer ratings for
mapping. Susceptible to friendship bias and sabotage. Does
not take utility of raters into consideration.
Incomplete contract, subject to appeal to higher academic
administration.

30. Discussion and Conclusion
The meaning of SPA is unknown therefore can not be used in
assigning marks.
SPA may distort the level of effort.
A co-operative game theoretic approach in assessment design
may give better results.

31. Thank You