Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Using Comparative Judgement to Assess Student e-Portfolios

1,849 views

Published on

Presentation given by Matt Wingfield (TAG) and Susan McLaren (University of Edinburgh) at eAssessment Scotland 2010

Published in: Education
  • Be the first to comment

  • Be the first to like this

Using Comparative Judgement to Assess Student e-Portfolios

  1. 1. Using Comparative Judgement to Assess Student e-Portfolios Matt Wingfield Managing Director – TAG Developments Susan McLaren Senior Lecturer - University of Edinburgh
  2. 2. The evolution of TAG’s e-portfolio solutions <ul><li>Portfolio Builder (1997) – developed by TAG in partnership with TERU specifically for The London Borough of Newham and Apple; </li></ul><ul><li>Red Pen Tool (2010) – on screen annotation/marking of any digital file; </li></ul><ul><li>e-scape (2004 - present) – Formative assessment or project based learning and delivery controlled qualification coursework assessment ; </li></ul><ul><li>MAPS (2001 - present) – Multi award-winning National Formative and Summative Assessment ePortfolio for curriculum and qualification work; </li></ul><ul><li>LiveAssess (2010) – Dynamically capture 21st Century Skills. </li></ul>
  3. 3. <ul><li>Utilised MAPS e-portfolio - enabling teachers to create, deliver, manage controlled assessments of student work (primary, secondary and H.E.) ; </li></ul><ul><li>Each student is given a portable device (PDA, Fizzbook Spin); </li></ul><ul><li>These devices are connected wirelessly to either a web-based or locally hosted e-portfolio system; </li></ul><ul><li>The students receive instructions on what they should do for each activity directly to their device; </li></ul><ul><li>As each activity is completed, the work, comments and evidence is dynamically captured by the e-portfolio to be viewed by the teacher. </li></ul><ul><li>Pilot project, run by Goldsmiths, University of London in England, University of Edinburgh in Scotland, and Limerick University in Ireland; </li></ul>Capturing Project Based Learning Introducing Project e-Scape
  4. 4. <ul><li>How does it work: </li></ul><ul><ul><li>An activity authoring tool is used to define the content and number of stages within the assessment; </li></ul></ul><ul><ul><li>The students access the activity through a web-based client; </li></ul></ul><ul><ul><li>Evidence is captured on the fly using in-built tools, including: </li></ul></ul><ul><ul><ul><li>text & drawing capture; </li></ul></ul></ul><ul><ul><ul><li>photo & video capture; </li></ul></ul></ul><ul><ul><ul><li>audio capture; and </li></ul></ul></ul><ul><ul><ul><li>mind maps . </li></ul></ul></ul><ul><ul><li>The Evidence is dynamically sent back to the storyboard portfolio for review. </li></ul></ul>Capturing Project Based Learning Introducing Project e-Scape
  5. 5. <ul><ul><li>Real-time capture of student evidence – progressive storyboard format; </li></ul></ul><ul><ul><li>Controlled assessment – structured, timed, no need for other software; </li></ul></ul><ul><ul><li>Ideal for supporting the assessment of 21 st Century Skills; </li></ul></ul><ul><ul><li>BUT – how do we assess creativity, team work or problem solving? </li></ul></ul>Capturing Project Based Learning Introducing Project e-Scape
  6. 6. Assessment vs. Evaluation Comparative Pairs Assessment <ul><li>What is Comparative Pairs? </li></ul><ul><li>Measuring without scales; </li></ul><ul><li>Based on Rasch/Thurstone - Law of Comparative Judgement ; </li></ul><ul><li>More recent work from Alastair Pollitt at the University of Cambridge; </li></ul><ul><li>Let’s try Comparative Pairs… </li></ul>
  7. 7. Assessment vs. Evaluation Comparative Pairs Assessment portfolio A portfolio B TAG has created a web-based comparative pairs engine, that facilitates highly accurate paired assessment of e-portfolios for the first time on a national scale!
  8. 8. The Comparative Pairs Engine dynamically refines the ranking of the student work through a series of judging rounds. What emerges is a collective professional consensus of an expert group of judges. Produces a reliability coefficient of 0.95 Assessment vs. Evaluation Comparative Pairs Assessment
  9. 9. <ul><li>UK: </li></ul><ul><ul><li>Continued research by Goldsmiths (Apprenticeships, Diplomas, SLTs) </li></ul></ul><ul><ul><li>Bath Spa University & AstraZeneca Science Teaching Trust to support science & technology in Primary schools </li></ul></ul><ul><li>Ireland: </li></ul><ul><ul><li>Limerick University to support Degree courses </li></ul></ul><ul><li>Israel: </li></ul><ul><ul><li>ORT - pilot of school-based examinations – esp. practical science </li></ul></ul><ul><li>Western Australia: </li></ul><ul><ul><li>Curriculum Council piloting the school leaving exams in Engineering/LOTE/IT and Physical Education </li></ul></ul>Comparative Pairs Assessment of e-portfolios: where is it being used
  10. 10. Contact Information & Useful Links Matt Wingfield Managing Director – TAG Developments [email_address] Tel: 07850 488 502 Susan McLaren Senior Lecturer – University of Edinburgh [email_address] 0131 651 6615 http://www.mapsassessment.com http://www.tagdevelopments.com

×