Presentation made at the ICEM&SIIE 2011, join conference, Universidade de Aveiro

1. E-assessment in collaborative blended
learning: evolving practices and students’
perceptions
Loureiro, MªJoão, Pombo, L. Balula, A. Moreira, A.
Loureiro et al. (2011). E-assessment in collaborative blended learning

2. Sumary
• problem/research questions
• context
• user generated content experience
– objectives
– methodology
• results
• final considerations
Loureiro et al. (2011). E-assessment in collaborative blended learning

3. • problem
policy/management
practice interaction research
how to improve?
Loureiro et al. (2011). E-assessment in collaborative blended learning

4. focus bLearning
•nature
•teaching strategies
•learning Online
•e-assessment
context
impact
practices (evolve,
improve, innovate)
research
Loureiro et al. (2011). E-assessment in collaborative blended learning

5. • evolution/research questions
EES DE DE
(MsD) (PhD 2008/09) (PhD 2010711)
evauation of teaching evauation of PA activities evauation of PA activities
(students’ perceptions) assessment of coll. work assessment of coll. work
(students’ perceptions) (students’ perceptions)
quality of peer assessment
Loureiro et al. (2011). E-assessment in collaborative blended learning

6. – PhD program in Multimedia in Education module
• Distance Education (DE) – two editions (2008/09 and 2010/11)
– blearning regime
• face-to-face and at a distance
• group work
– DE module aim - develop research competences –
• search, select, systematize, synthesize and literature related with DE;
• communication, collaborative work and assessment competences (self-
and peer- assessment)
“good enough doctorate” (Wisker, 2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning

7. – PA benefits:
• promote higher order thinking, developing students’ critical
thinking, communication, lifelong learning, problem solving…
• increase the amount of feedback, from teachers and from peers
• enhance students’ sense of ownership, responsibility and students’
motivation, since they find it useful, attractive and enjoyable
• support active and autonomous learning
• can avoid the involvement of free-riders in group work, i.e,
facilitate the identification of individual contributions
Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010),
Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),
Zundert et al. (2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning

8. – PA pitfalls:
• students’ attitudes toward PA may hinder the above mentioned
benefits – unconfidence towards PA –> resistant toward PA
• PA can be subjective – students might be too hard, critical or
boasting, or not used to assess their peers -> increase students’
anxiety
• PA is time-consuming – training, preparation, and monitoring are
needed
– PA advantages increase – formative feedback
Boud and Falchikov (2007), Joordens et al. (2009), Li et al. (2009, 2010), Peng (2010),
Rourke et al. (2008), Sluijsmans et al. (2004), Topping (2000, 2008, 2010),
Zundert et al. (2010)
Loureiro et al. (2011). E-assessment in collaborative blended learning

9. • User Generated Content (UGC) experience and methodology
– case studies (blearning courses/user generated content – literature
review ODL) / action research (reflexion on/after action).
– strategies - project work, problem solving, collaborative learning and
assessment for learning -> quality in online learning.
– duration of the DE module, participants & tools:
• 2008/09: five-week period – 24 students – Wiki (
http://ead0809.wetpaint.com), Google Docs, EndNote Web…;
• 2010/11: four weeks) – 18 students – Ning (http://
labconteudosdigitais.ning.com), Google Docs, EndNote Web…
Loureiro et al. (2011). E-assessment in collaborative blended learning

10. • changes in 2010/11 edition:
– provided extra opportunities for shared PA and use of
assessment criteria of the group work
• it was used for formative e-assessment of three versions of
the ongoing work instead of one;
– assignment made by email -> confidentiality -> authenticity.
Loureiro et al. (2011). E-assessment in collaborative blended learning

11. • results (difficulty and relevance of PA activities)
2008/09 Academic Year!
2008/09 20010/11
2010/11 Academic Year
Survey of perceptions about how to make a Survey of perceptions about how to make a
literature search! literature search…
Partial and final individual reflection (self-
Weekly workgroup reflexive analysis
assessment) !
Self and peer-assessment of the group work and of Self and peer-assessment of the group work
the develloped competences! and of the develloped competences
0%! 10%! 20%! 30%! 40%! 50%! 60%! 70%! 80%! 90%! 100%! 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Difficulty! Relevance! Difficulty Relevance
Loureiro et al. (2011). E-assessment in collaborative blended learning

12. • results (privacy, participants and components )
Academic Year2008/09
- 2008/09 20010/11
Academic Year - 2010/11
Self and peer assessment of developed competences Self and peer assessment of developed competences
(inside each group) should be published in the (inside each group) should be published in the
module's site module's site
Self and peer assessment of the group works should Self and peer assessment of the group works should
be published in the module's site be published in the module's site
Collabarative work should be assessed only by Collabarative work should be assessed only by
teachers teachers
Collabarative work should be assessed individuall, Collabarative work should be assessed individually,
by teachers and by colleagues by teachers and by colleagues
Collabarative work should beassessed with several Collabarative work should beassessed with several
components components
Collabarative work should be assessed as a whole Collabarative work should be assessed as a whole
with the same mark for all elements with the same mark for all elements
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% 0% 10% 20% 30% 40% 50%60% 70% 80% 90%100%
completely disagree disagree agree completely agree
completelydon't know disagree
disagree agree completely agree don't know
Loureiro et al. (2011). E-assessment in collaborative blended learning

13. • final considerations - connections with UGC state of art
1. definition of UGC
 content created by postgraduate students (literature reviews);
 shared freely;
 content has not been evaluated through a process of formal peer
review (journals);
 students performed self-reflexive task concerning the groupwork
and the developed competencies.
2. quality in elearning
– activities fit the students’ needs and enriched learning context,
active involvement in the production of a literature review
(product and processes).
Loureiro et al. (2011). E-assessment in collaborative blended learning

14. • final considerations - relations with the CONCEDE framework
– CONCEDE framework for peer creation - peer-validation
• writing process involved several activities: authoring, editing,
updating, revising, reflecting…;
• DE module has been evaluated by different stakeholders:
– students;
– university professor not involved in the module - peer-evaluation;
– formal peer-assessment - submitted for publication (journals and
international conferences, the experience has been evaluated at the first
and second level of the quality pyramid.
Loureiro et al. (2011). E-assessment in collaborative blended learning

15. • final considerations
– limitations - personal experience, few contexts, number of
students/comparability, qualitative approach…
– further work
• content analysis (PA, reflexive activities, products…), triangulation
• share, dissemination -> transfer to other context
– old - improve students learning - meets new???
• does the available ICT tools help, how, in what circumstances?
• is its use sustainable?
– environment, economical, social development impact???
– inclusion or exclusion??)
Loureiro et al. (2011). E-assessment in collaborative blended learning

16. questions and comments???
mjoao@ua.pt
lpombo@ua.pt
balula@ua.pt
moreira@ua.pt
Loureiro et al. (2011). E-assessment in collaborative blended learning

17. • results (1th edition DE) – quality of PA
Adequacy of the final considerations
Fairness of the score
Provision of constructive feedback
(suggestions for improvement)
Provision of constructive feedback
(questions)
Provision of constructive feedback
(criticisms)
Adequacy of the chosen vocabulary
Use of the negotiated criteria
0 20 40 60 80 100 %
Loureiro et al. (2011). E-assessment in collaborative blended learning

18. • module organisation
modules organisation (DocProg)
Loureiro et al. (2011). E-assessment in collaborative blended learning