Using Comparative Judgement to Assess Student e-Portfolios Matt Wingfield Managing Director – TAG Developments Susan McLar...
The evolution of TAG’s e-portfolio solutions <ul><li>Portfolio Builder (1997)  – developed by TAG in partnership with TERU...
<ul><li>Utilised MAPS e-portfolio - enabling teachers to create, deliver, manage controlled assessments of student work (p...
<ul><li>How does it work: </li></ul><ul><ul><li>An activity authoring tool is used to define the content and number of sta...
<ul><ul><li>Real-time capture of student evidence – progressive storyboard format; </li></ul></ul><ul><ul><li>Controlled a...
Assessment vs. Evaluation Comparative Pairs Assessment <ul><li>What is Comparative Pairs? </li></ul><ul><li>Measuring with...
Assessment vs. Evaluation Comparative Pairs Assessment portfolio A portfolio B TAG has created a web-based comparative pai...
The Comparative Pairs Engine  dynamically  refines the ranking of the student work  through a series of judging rounds. Wh...
<ul><li>UK: </li></ul><ul><ul><li>Continued research by Goldsmiths (Apprenticeships, Diplomas, SLTs) </li></ul></ul><ul><u...
Contact Information & Useful Links Matt Wingfield Managing Director – TAG Developments [email_address] Tel: 07850 488 502 ...
Upcoming SlideShare
Loading in …5
×

Using Comparative Judgement to Assess Student e-Portfolios

1,745 views
1,647 views

Published on

Presentation given by Matt Wingfield (TAG) and Susan McLaren (University of Edinburgh) at eAssessment Scotland 2010

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,745
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Using Comparative Judgement to Assess Student e-Portfolios

  1. 1. Using Comparative Judgement to Assess Student e-Portfolios Matt Wingfield Managing Director – TAG Developments Susan McLaren Senior Lecturer - University of Edinburgh
  2. 2. The evolution of TAG’s e-portfolio solutions <ul><li>Portfolio Builder (1997) – developed by TAG in partnership with TERU specifically for The London Borough of Newham and Apple; </li></ul><ul><li>Red Pen Tool (2010) – on screen annotation/marking of any digital file; </li></ul><ul><li>e-scape (2004 - present) – Formative assessment or project based learning and delivery controlled qualification coursework assessment ; </li></ul><ul><li>MAPS (2001 - present) – Multi award-winning National Formative and Summative Assessment ePortfolio for curriculum and qualification work; </li></ul><ul><li>LiveAssess (2010) – Dynamically capture 21st Century Skills. </li></ul>
  3. 3. <ul><li>Utilised MAPS e-portfolio - enabling teachers to create, deliver, manage controlled assessments of student work (primary, secondary and H.E.) ; </li></ul><ul><li>Each student is given a portable device (PDA, Fizzbook Spin); </li></ul><ul><li>These devices are connected wirelessly to either a web-based or locally hosted e-portfolio system; </li></ul><ul><li>The students receive instructions on what they should do for each activity directly to their device; </li></ul><ul><li>As each activity is completed, the work, comments and evidence is dynamically captured by the e-portfolio to be viewed by the teacher. </li></ul><ul><li>Pilot project, run by Goldsmiths, University of London in England, University of Edinburgh in Scotland, and Limerick University in Ireland; </li></ul>Capturing Project Based Learning Introducing Project e-Scape
  4. 4. <ul><li>How does it work: </li></ul><ul><ul><li>An activity authoring tool is used to define the content and number of stages within the assessment; </li></ul></ul><ul><ul><li>The students access the activity through a web-based client; </li></ul></ul><ul><ul><li>Evidence is captured on the fly using in-built tools, including: </li></ul></ul><ul><ul><ul><li>text & drawing capture; </li></ul></ul></ul><ul><ul><ul><li>photo & video capture; </li></ul></ul></ul><ul><ul><ul><li>audio capture; and </li></ul></ul></ul><ul><ul><ul><li>mind maps . </li></ul></ul></ul><ul><ul><li>The Evidence is dynamically sent back to the storyboard portfolio for review. </li></ul></ul>Capturing Project Based Learning Introducing Project e-Scape
  5. 5. <ul><ul><li>Real-time capture of student evidence – progressive storyboard format; </li></ul></ul><ul><ul><li>Controlled assessment – structured, timed, no need for other software; </li></ul></ul><ul><ul><li>Ideal for supporting the assessment of 21 st Century Skills; </li></ul></ul><ul><ul><li>BUT – how do we assess creativity, team work or problem solving? </li></ul></ul>Capturing Project Based Learning Introducing Project e-Scape
  6. 6. Assessment vs. Evaluation Comparative Pairs Assessment <ul><li>What is Comparative Pairs? </li></ul><ul><li>Measuring without scales; </li></ul><ul><li>Based on Rasch/Thurstone - Law of Comparative Judgement ; </li></ul><ul><li>More recent work from Alastair Pollitt at the University of Cambridge; </li></ul><ul><li>Let’s try Comparative Pairs… </li></ul>
  7. 7. Assessment vs. Evaluation Comparative Pairs Assessment portfolio A portfolio B TAG has created a web-based comparative pairs engine, that facilitates highly accurate paired assessment of e-portfolios for the first time on a national scale!
  8. 8. The Comparative Pairs Engine dynamically refines the ranking of the student work through a series of judging rounds. What emerges is a collective professional consensus of an expert group of judges. Produces a reliability coefficient of 0.95 Assessment vs. Evaluation Comparative Pairs Assessment
  9. 9. <ul><li>UK: </li></ul><ul><ul><li>Continued research by Goldsmiths (Apprenticeships, Diplomas, SLTs) </li></ul></ul><ul><ul><li>Bath Spa University & AstraZeneca Science Teaching Trust to support science & technology in Primary schools </li></ul></ul><ul><li>Ireland: </li></ul><ul><ul><li>Limerick University to support Degree courses </li></ul></ul><ul><li>Israel: </li></ul><ul><ul><li>ORT - pilot of school-based examinations – esp. practical science </li></ul></ul><ul><li>Western Australia: </li></ul><ul><ul><li>Curriculum Council piloting the school leaving exams in Engineering/LOTE/IT and Physical Education </li></ul></ul>Comparative Pairs Assessment of e-portfolios: where is it being used
  10. 10. Contact Information & Useful Links Matt Wingfield Managing Director – TAG Developments [email_address] Tel: 07850 488 502 Susan McLaren Senior Lecturer – University of Edinburgh [email_address] 0131 651 6615 http://www.mapsassessment.com http://www.tagdevelopments.com

×