Keith Willey & Anne Gardner 2008a


Published on

Session B - Bradley

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Keith Willey & Anne Gardner 2008a

  1. 1. Using Self and Peer Assessment for Professional and Team Skill Development Do well functioning teams experience the benefits? Keith Willey & Anne Gardner University of Technology, Sydney
  2. 2. Introduction <ul><li>Self and peer assessment effective in promoting development of professional skills including teamwork </li></ul><ul><li>In previous research approximately 30% of students were neutral as to whether it improved their teamwork experience. </li></ul><ul><li>Hypothesised these students probably members of well functioning team </li></ul><ul><ul><li>functioned well without self and peer assessment </li></ul></ul><ul><ul><li>hence did not improve their teamwork experience. </li></ul></ul><ul><li>This paper reports testing this hypothesis and finding it to be incorrect . </li></ul>
  3. 3. Background: Competency Gap <ul><li>In addition to technical competence, professionals require skills of collaboration, communication and the ability to work in teams </li></ul><ul><li>Technical expertise, typically receives the greatest amount of teaching time during degree </li></ul><ul><li>Not surprisingly competency gap reported between professional skills required by employers and those developed by students during undergraduate courses </li></ul><ul><li>Universities identify learning outcomes (graduate attributes) </li></ul><ul><li>With imagination Self and peer assessment can be effective in promoting the development of just about any skill. </li></ul>
  4. 4. Team Based Projects <ul><li>Provide opportunities but don’t necessarily develop teamwork skills </li></ul><ul><li>Students need to: </li></ul><ul><ul><li>Understand team dynamics </li></ul></ul><ul><ul><li>How to resolve conflict </li></ul></ul><ul><ul><li>How to give and receive feedback </li></ul></ul><ul><li>Instruction, insufficient, need learning-oriented assessments </li></ul><ul><ul><li>opportunities to practice </li></ul></ul><ul><ul><li>incentive for skills to be developed </li></ul></ul><ul><ul><li>promote ongoing development and learning </li></ul></ul><ul><ul><li>feedback that promotes outcomes. </li></ul></ul>
  5. 5. Feedback <ul><li>Arguably, part of assessment process with most potential to affect future learning and student achievement </li></ul><ul><li>To be productive and used for reflection, must be both timely and focused. </li></ul><ul><li>Self and peer assessment has potential to address these issues but are we getting it right. </li></ul>
  6. 6. SPARK Tool <ul><li>SPARK requires participants to rate each other over criteria </li></ul><ul><li>Recommend including criteria for both discipline specific and demonstration of professional skills e.g. good team practices </li></ul>
  7. 7. SPARK Screen Shot
  8. 8. Self and Peer Assessment <ul><li>Authors use self and peer assessment to promote development of discipline specific and professional skills and facilitate the provision of regular feedback in large engineering classes. </li></ul><ul><li>In previous research found that it: </li></ul><ul><ul><li>improved group work experience </li></ul></ul><ul><ul><li>reduced free-riding </li></ul></ul><ul><ul><li>encouraged professional skill development </li></ul></ul><ul><li>Students reported that criteria to specifically assessed teamwork processes: </li></ul><ul><ul><li>Encouraged team cooperation </li></ul></ul><ul><ul><li>Commitment </li></ul></ul><ul><ul><li>increased engagement . </li></ul></ul>
  9. 9. Previous Research <ul><li>Authors consistently found approx 30% of students neutral as to whether using self and peer assessment had improved their group work experience. </li></ul><ul><li>Hypothesised that a significant number of these students were probably members of well functioning teams </li></ul><ul><li>we tested this hypothesis and investigated whether students in well functioning teams benefit as much from self and peer assessment processes as those in teams with at least one poor team member. </li></ul>
  10. 10. Implementation <ul><li>Design Fundamentals Stage 3 compulsory core subject within the engineering degree at the University of Technology, Sydney. </li></ul><ul><li>Typical cohort 280 - 320 students </li></ul><ul><li>Tutorial classes of 32 students. </li></ul><ul><li>Multistage team project (50% or grade) </li></ul><ul><li>Intentional focus on provision of feedback </li></ul><ul><li>To develop professional skills, provide feedback, practice and ongoing learning encourage, academic honesty, self and peer assessment used 3 (now 4) times during semester. </li></ul>
  11. 11. Objective <ul><ul><li>provide feedback on discipline and / or teamwork skills and contribution to team. </li></ul></ul><ul><ul><li>develop students’ critical evaluation and feedback skills. </li></ul></ul><ul><ul><li>identifying individual strengths and weaknesses. </li></ul></ul><ul><ul><li>provide opportunity to respond to feedback to improve subsequent performance. </li></ul></ul><ul><ul><li>move students from being novice to be more expert in their professional skill development (Boud) </li></ul></ul><ul><ul><li>determine individual assignment marks by appropriate adjustment to group marks. </li></ul></ul>
  12. 12. Feedback Process <ul><li>All team members receive both factors for each peer </li></ul><ul><li>Guided to reflect own performance & development and give constructive feedback to peers. </li></ul><ul><ul><ul><li>reflect </li></ul></ul></ul><ul><ul><ul><li>provide positive feedback </li></ul></ul></ul><ul><ul><ul><li>share own self-evaluation </li></ul></ul></ul><ul><ul><ul><li>Provide feedback how peers could improve performance. </li></ul></ul></ul><ul><li>Teams agree how to improve overall and individual performance in future team activities </li></ul>
  13. 13. Method <ul><li>Autumn 2007 post-subject survey conducted </li></ul><ul><li>95 students from cohort 220 agreed to complete questionnaire. </li></ul><ul><li>While not specifically designed for purpose we re-analysed data to compare the experience of students in teams with and without any poor team members. </li></ul><ul><li>investigated how many students who responded ‘neutral’ to improved group work experience indicated they had no poor team members. </li></ul><ul><li>compare the responses of students with and without poor team members, to investigate whether there is a link between benefit students received and how well their team functioned. </li></ul>
  14. 14. Results <ul><li>Again approx 30% ‘neutral’ as to whether self and peer assessment had improved their group work experience. </li></ul><ul><li>Only 6 (22%) of these students also reported having no poor team members </li></ul><ul><li>Furthermore neutral response for student in groups with at least one poor team member (32%) larger than for those in well functioning teams (20%) </li></ul><ul><li>Suggest neutral respondents cannot be assumed to be primarily members of well functioning teams. </li></ul>
  15. 15. Results con’t <ul><li>30 respondents (cohort 95) reported having no poor team members. </li></ul><ul><li>To investigate whether students in well functioning teams receive same benefit from using self and peer assessment as those in teams with at least one poor team member we compared responses for each group. </li></ul><ul><li>Overall the results for both groups remarkably similar. </li></ul>
  16. 17. Discussion <ul><li>Many students focus on free-rider deterrent aspect, those in good teams saying had little to discuss in the feedback session as everyone in the team ‘pulled’ their weight. </li></ul><ul><li>Typically did not take opportunity to discuss how they could have done better or improved their project </li></ul><ul><ul><li>hence missed opportunity to benefit from feedback to assist ongoing professional development. </li></ul></ul><ul><li>Givn result can not ignore 46% of students reporting self and peer assessment did not improve their group work experience. </li></ul><ul><li>Unlikely to improve, while students continue to perceive self and peer assessment as instrument to facilitate fairness, </li></ul>
  17. 18. Recommendations <ul><li>In an effort to increase both student engagement and the benefit received we have introduced changes; </li></ul><ul><li>Training for tutors, provide language to enable tutors to better explain value of self and assessment and how to interpret the results. </li></ul><ul><ul><li>Role play </li></ul></ul><ul><ul><li>Practice </li></ul></ul><ul><ul><li>Support </li></ul></ul><ul><li>Students actively encouraged to view using self and peer assessment as a learning opportunity, to provide feedback to assist professional development, learning and help team to produce a better project. </li></ul>
  18. 19. Recommendations <ul><li>After each student’s SPA and SAPA factors shared with all team members, groups are guided through a feedback process. </li></ul><ul><ul><li>students share positive feedback focus on what their peers did well and what they learnt from their peers. </li></ul></ul><ul><ul><li>self evaluation students share with group what they learnt or discovered about their strengths, weaknesses or performance from the exercise. </li></ul></ul><ul><ul><li>Students asked to suggest how others in their group may have approached tasks differently to achieve better group result </li></ul></ul><ul><ul><li>Students share what they consider to be the weaker aspects of a peer’s contribution and how this could have been improved. </li></ul></ul><ul><ul><li>concludes by team agreeing how to improve their overall team and individual performance </li></ul></ul>
  19. 20. Conclusion <ul><li>While reported data needs to be treated with caution it suggests that the benefit students gain from self and peer assessment processes is a function of how each individual student engages with these processes rather than how well their team functioned. </li></ul><ul><li>We found that some students focused on the free-rider deterrent aspect of using self and peer assessment. </li></ul><ul><ul><li>Typically these students don’t take opportunity to benefit from reflective nature of self and peer assessment and feedback it provides. </li></ul></ul><ul><li>For students to receive these benefits we must do more to ensure students engage with the process. </li></ul><ul><li>First step recommend feedback sessions be focused on learning and not just assessment outcomes. Provide better training and support for staff. </li></ul>