Review of Advanced e-Assessment Techniques Project by Martin Ripley Ltd (the team: Martin Ripley; Jeremy Tafler; Hakan Redif; Robert Harding; Mike Peppiatt; Jim Ridgway) Email contact: Martin Ripley on martin.ripley1@btinternet.com

Review of Advanced e-Assessment Techniques Project Purpose: Build a significant body of information about who is using different techniques, the associated issues and benefits The project will: Identify a range of projects/techniques useful to HE and FE; Select and study 5-10 case studies to obtain information in: Test and assessment systems; Task/test design; Administration; Marking, scoring and making judgements.

Project Purpose:

Build a significant body of information about who is using different techniques, the associated issues and benefits

The project will:

Identify a range of projects/techniques useful to HE and FE;

Select and study 5-10 case studies to obtain information in:

Test and assessment systems;

Task/test design;

Administration;

Marking, scoring and making judgements.

Review of Advanced e-Assessment Techniques Current state of play: Over 90 projects identified Working with the University of Durham to create the online ‘Advanced e-Assessment Catalogue’ URL = www.dur.ac.uk/smart.centre1/aeac Pages likely to be active by the end of February 2009. Final project report to be delivered by end of March 2009. An example entry ‘in the catalogue’: Application LISC Site Kent University Contact name Alison Fowler Publications caa 2008 Email [email_address] Description CALL system allowing input of whole phrases: free text scoring for MFL marked by evaluating language strings; significantly different approach to NLP

Current state of play:

Over 90 projects identified

Working with the University of Durham to create the online ‘Advanced e-Assessment Catalogue’

URL = www.dur.ac.uk/smart.centre1/aeac

Pages likely to be active by the end of February 2009.

Final project report to be delivered by end of March 2009.

An example entry ‘in the catalogue’:

Review of Advanced e-Assessment Techniques Question: What to do with the list of the 90+ projects? Issues: Potential for something more dynamic Usefulness for the JISC community Tool/resource for further use Interactive/management/updating [email_address]

Question:

What to do with the list of the 90+ projects?

Issues:

Potential for something more dynamic

Usefulness for the JISC community

Tool/resource for further use

Interactive/management/updating

[email_address]

Review of Advanced e-Assessment Techniques The case studies chosen from the list: Broad area Candidate Adaptive testing AsTTle, University of Auckland Combining human/ computer marking Assessment 21, University of Manchester Higher order skills Premium, Federation of State Medical Boards of the United States Language testing LISC, Kent University Versant (with Ordunate technology), Psyhcorp, USA Short text* Automark, Dundee University IAT, Open University Storage/databanks* A range of practices (no specific focus on 1 candidate)

The case studies chosen from the list:

Review of Advanced e-Assessment Techniques Emerging findings (not conclusions): Isolated nature of innovations leads to difficulties in ‘growing’ the concept for wider use. Due to isolated nature of some developments, technology to support cross-institutional applications not exploited. The crucial need for applications to demonstrate business efficiencies (e.g. saving tutors’ time) to win wider adoption. Lack of consistency in adoption of key standards. Nervousness of take up of new technologies vs willingness of institutions governing bodies to support e-Assessment.

Emerging findings (not conclusions):

Isolated nature of innovations leads to difficulties in ‘growing’ the concept for wider use.

Due to isolated nature of some developments, technology to support cross-institutional applications not exploited.

The crucial need for applications to demonstrate business efficiencies (e.g. saving tutors’ time) to win wider adoption.

Lack of consistency in adoption of key standards.

Nervousness of take up of new technologies vs willingness of institutions governing bodies to support e-Assessment.