This reports on a project that investigated the use of digital badges to support peer assessment.

1. O P E N B A D G E S T O S U P P O R T
P E E R F O R M A T I V E A S S E S S M E N T
I N O N L I N E L E A R N I N G E N V I R O N M E N T S
S T Y L I A N O S H A T Z I PA N A G O S , K I N G ’ S C O L L E G E L O N D O N &
S T E V E N W A R B U RTO N , U N I V E R S I T Y O F S U R R E Y

2. CONTEXT
A project that explored the effectiveness of peer feedback and assessment:
Investigated whether open badges can support learning in online environments
through enhanced engagement;
Revisited and evaluated badge typologies with the aim of identifyingthe most
appropriate typology that supports peer feedback and assessment;
And created a clear correspondence between the typology of badges and peer
feedback and assessment practice in online classrooms.

3. RATIONALE
Use of digital badges to support peer
feedback and assessment in higher education.
By the use of open badges in encouraging and
rewarding competences, knowledge and skills
acquired in digital learning spaces, we aimed
to reinforce the value of peer assessment and
by extension peer feedback and review
activities in supporting student learning.

4. PEER ASSESSMENT DEFINITION
Peer assessment is to be understood as an educational
arrangement in which students assess the quality of their
fellow students’ work and provide one another with
feedback.
Dochy et al., 1999

5. FORMATIVE E-ASSESSMENT CAN:
• engage learners with tasks that enhance learning outcomes and test
the development of higher order capabilities;
• enrich learning approaches by making feedback central to all
assessment activities;
• promote a dialogue in relation to feedback, peer and self
assessment activities.
Hatzipanagos & Warburton 2009

6. PEER ASSESSMENT/PEER REVIEW
• Peer assessment is frequently underused in online learning
environments.
• Its informal learning value is not exploited by peers as it is not
explicitly linked to summative assessment activities.
• There is a reluctance to participate in such peer review activities
and an ambivalence of tutors in relation to their value.
• Informal aspect is not clearly or meaningful embedded in the
learning process.

7. Peer assessment
attributes (Dochy
et al)
What the research says Badges
Validity Peers are prone to produce ratings based on uniformity,
race and friendship if there is no extensive training in peer
rating.
Majority of studies showed an acceptably high validity
and reliability in a variety
of fields.
Clear criteria embedded in badges definition.
Validity can be high if badges are well defined.
Fairness. Student perceptions of fairness of peer assessment
Students were very positive about the effects of
alternative assessment on their learning.
Openness and clarity in microcertification
fundamental requirements of a fair and valid
assessment
system.
Accuracy. Agreement between student and tutors.
No consistent results.
There can be ‘objective’ approaches of
testing accuracy in microcertification
Effects. Peer assessment fostered an appreciation for the
individuals' performance within the group and
interpersonal relationships in the classroom
Badges can capture interpersonal
relationships (soft skills) and performance
(hard skills)
Implications for
practice.
valuable as a formative assessment method but…
friendship marking; collusive marking; decibel
marking; and parasite marking.
.
Appropriateness of emergent technologies
such as badges for formative assessment
Assessing ‘mishaps’ can be prevented by
learning analytics and digital trail in new
environments

8. METHODOLOGY
The Open Badges used in King’s College London and the University of Surrey.
King’s programme (Postgraduate Certificate in Academic Practice) targets novice
lecturers,PhD students with a teaching role,librarians and information specialists
who have chosen to study a ‘Supporting technology enhanced learning’ module.
PeerAssessment is an integral part of these modules and learners are strongly
encouraged to participate in formative assessment activities.There was an
opportunity to link performance and peer review in these activities with open
badges.

9. METRICS
Assigning badges was a semi-automated process.
1. Use of the learning environmenttriggered automated assignment of
badges for metrics associated with use of the learning environment e.g.
when students used discussion fora and the social media communication.
Metrics such as number of posts to a discussion forum,number of log-ins,
response to other posts,sharing content and reflections were used.
2. Other badges were deployed by application and evidence of achievement,
moderated by the project team or by peers on the course.A protocol for
awarding a badge was developed by the project team in collaboration with
the module tutors and learners.
Evaluation of the project activities was carried out by collecting data from the
online learning environment,using focus group interviews and by end of module
and programme evaluation questionnaires.

10. PROJECTIONS
• By rewarding positive behaviours the participants would feel more motivated
to complete learning activities.The aim of this approach was to manage any
motivational drops that can commonly be experienced in online/blended
courses.
• Participants were encouraged to push their practice beyond the central
learning design journey and give opportunitiesfor digital professional identity
development.
• The project would provide a mechanism for self-reflection by the academics
engaged in the study.They would be able to evaluate the benefit of using a
badging system within their own learning designs.

11. CONCRETE EXAMPLES
• For example, the ability to award a badge to your peer evaluator if they provide good
constructive advice on your work.

12. Powered by
Digital Badges evaluation

13. Q2: HOW DO BADGES COMPARE WITH
OTHER WAYS OF DEMONSTRATING YOUR
ACHIEVEMENTS?

14. Q5: DO YOU THINK THAT BADGES
MOTIVATE LEARNERS IN AN ONLINE
ENVIRONMENT?

15. Q14: DO YOU SEE BADGES BECOMING
PART OF YOUR DIGITAL ACADEMIC
IDENTITY?

16. PERCEPTIONS OF VALUE
• Random orchestrations between learners that are not part of the same
community.
‘Due to the limited time I spent with my fellow students I didn't get to know them well.
Therefore I felt that the use of badges was not particularly useful’
• Soft indicator of achievement, used for formative rather than summative purposes.
‘in lieu of marking or registers they could be used as a 'soft' indicator of achievement (e.g. a
record of optional on-line courses taken)’
• Overall caution against the use of badges
‘don't think the possibility of winning a digital badge really changes adult behaviour’
‘unless digital badges are established to international- or the very least national standards as a
well recognized metric of achievement I wouldn't think they will affect online learning.’
• Student centred approach should work
‘something that the students themselves defined and awarded, rather than it coming from me
(also a good way to gauge what they think of the whole thing!)’

17. WHAT DO YOU THINK WOULD HELP THE
ADOPTION OF BADGES?
• I think a clear rationale for the benefits to both the student and staff
• Defining their purpose more clearly, and applying them consistentlyto something that is meaningful to students' learning and
achievements.Instinctively,it's quite hard to take them seriously.Is it the name? or the varied application? Not sure.
• If someone badges feed into the assessment process -i.e.maybe 5 extra marks could be added to the summative assessment to
recognize participation and contribution.
• If they were linked to module credit
• As the criteria for digital badges are still very loose I don't think it is likelythat they are used in evaluating academic contribution.
Stricter and well defined metrics are in place for academic professional development (at least in Life Sciences).
• I might be wrong about this - and it's a bit of a chicken and egg situation - but I don’t see them as an important part of a professional
identity,and I gather fromfriend on corporate e industry (e.g. cloud computing) that they are not taken seriouslythere either.If I felt
that they were something that employment panels cared about and that validated my use of e-learning tools,then I'd swallow my
scepticism,adopt them enthusiasticallyand show off about the ones I have earned.Of course,if none of us decide to adopt that
approach then the panels won't care, and so on around the circle...it's just really difficult for me not to see them as something rather
childish and believe that my students would feel the same.
• I think there's potential use for badges as part of the summative assessment (which would then require further guidelines and
transparency).
• Academia is already a game measured by the 'badges' of qualifications,publications,citations and research grants -- further gamification
seems unnecessary

18. CONCLUSIONS
• Badges as a ‘Trojan horse’ for recognizing and embedding
transversal skills in university curricula;
• Perception of value of badges for summative assessment;
promotion; recruitment and employability;
• Establishing professional identity linked to generational attitudes;
(and digital literacies?),‘good for our students but not good for us’;
• Motivation is not a key driver in using badges, at least for teaching
practitioners.

19. RECOMMENDATIONS
• Integration of badges (and ePortfolios) with university systems (e.g.
VLE) is necessary for uptake;
• Consider value of badges for summative assessment; promotion;
recruitment;
• Integration of badges with external certification systems (Mozilla
open badges, Open Badge Factory) and these with social media;
• Embed micro-portfolios of badges in PDP (personal development
planning) and portfolio building for staff and students.

20. REFERENCES
• Dochy , F., Segers, M. & Sluijsmans,D. (1999)The use of self-, peer and co-assessment in higher
education:A review, Studies in Higher Education,24:3,331-350,
DOI:10.1080/03075079912331379935.
• Hatzipanagos, S.& Warburton,S. (Eds) (2009). Social Software and Developing Community
Ontologies. 598 pp.London: Information Science Reference, an imprint of IGI Global ISBN 978-
1-60566-208-4.

21. IF YOU HAVE
QUESTIONS OR FEEDBACK
Email
s.hatzipanagos@kcl.ac.uk