e-Assessment Initiatives Myles Danson JISC Programme Manager (e-Learning)

e-Assessment – Myles Myles – Who? My portfolio: T&S, CETIS, Programme Level Activities….. e-Assessment lead JISC – what’s it all about? http:// www.jisc.ac.uk/aboutus/strategy.aspx e-Learning Programme http://www.jisc.ac.uk/whatwedo/themes/elearning/programme_elearning.aspx

Myles – Who?

My portfolio:

T&S, CETIS, Programme Level Activities…..

e-Assessment lead

JISC – what’s it all about? http:// www.jisc.ac.uk/aboutus/strategy.aspx

e-Learning Programme http://www.jisc.ac.uk/whatwedo/themes/elearning/programme_elearning.aspx

e-Assessment – JISC Definition The end-to-end electronic assessment processes where ICT is used for the presentation of assessment and the recording of responses.

The end-to-end electronic assessment processes where ICT is used for the presentation of assessment and the recording of responses.

e-Assessment Examples Automated marking of Objective Questions Variants of Objective questions Confidence level Assessment Assertion Reason Numerical input Automated marking of non-objective questions Peer Assessment Simulations Free text marking Use of Web 2.0 technologies Scenarios Self assessment (assessment of learning for learning sake) Formal summative assessment (performance measure, norm or criterion referenced) Diagnostic assessment

Automated marking of Objective Questions

Variants of Objective questions

Confidence level Assessment

Assertion Reason

Numerical input

Automated marking of non-objective questions

Peer Assessment

Simulations

Free text marking

Use of Web 2.0 technologies

Scenarios

Self assessment (assessment of learning for learning sake)

Formal summative assessment (performance measure, norm or criterion referenced)

Diagnostic assessment

Related activities ePortfolios: Formal documentation of eassessment outcomes and results Informal personal portfolios for supporting learning and development Recording achievement and activities to support lifelong learning and continuing professional development Repositories: Storage of eassessment material in secure and open repositories Discovery, retrieval and reuse of assessment materials eAdmin: Integration with institutional reporting procedures Scheduling of assessment activities, room bookings, etc Security Attainment records supporting progression / life long learner Flexible infrastructure to support these eFramework: ( http:// www.elearning.ac.uk/news_folder/eframeworkpaper )

ePortfolios:

Formal documentation of eassessment outcomes and results

Informal personal portfolios for supporting learning and development

Recording achievement and activities to support lifelong learning and continuing professional development

Repositories:

Storage of eassessment material in secure and open repositories

Discovery, retrieval and reuse of assessment materials

eAdmin:

Integration with institutional reporting procedures

Scheduling of assessment activities, room bookings, etc

Security

Attainment records supporting progression / life long learner

Flexible infrastructure to support these

eFramework: ( http:// www.elearning.ac.uk/news_folder/eframeworkpaper )

National E-Assessment Drivers and Environment Flexibility Personalisation National Drivers and policies DfES unified e-learning strategy www.dfes.gov.uk/elearningstrategy / ‘Using technology to streamline assessment procedures and enable online assessment on-demand is a long term objective ’ 14-19 Government White Paper www.14-19reform.gov.uk ‘We will seek to maximise the potential of e-assessment’ QCA http://www.qca.org.uk/qca_5412.aspx ‘QCA are currently reviewing the aims for e-assessment in line with qualification and curriculum developments ’ QCA Regulatory Principles for e-Assessment (April 2007) http://www.qca.org.uk/qca_10475.aspx HEFCE Strategy for e-Learning (under review with JISC) http://www.hefce.ac.uk/pubs/HEFCE/2005/05_12/ ‘Encourage the use of technology to enable electronic assessment’

Flexibility

Personalisation

National Drivers and policies

DfES unified e-learning strategy www.dfes.gov.uk/elearningstrategy / ‘Using technology to streamline assessment procedures and enable online assessment on-demand is a long term objective ’

14-19 Government White Paper www.14-19reform.gov.uk ‘We will seek to maximise the potential of e-assessment’

QCA http://www.qca.org.uk/qca_5412.aspx ‘QCA are currently reviewing the aims for e-assessment in line with qualification and curriculum developments ’ QCA Regulatory Principles for e-Assessment (April 2007) http://www.qca.org.uk/qca_10475.aspx

HEFCE Strategy for e-Learning (under review with JISC) http://www.hefce.ac.uk/pubs/HEFCE/2005/05_12/ ‘Encourage the use of technology to enable electronic assessment’

JISC e-Assessment Objectives To explore and develop effective practice on the use of e-assessment systems and tools Through the development of standards and piloting of e-assessment related technologies To provide guidance for institutions on effective practice in this area

To explore and develop effective practice on the use of e-assessment systems and tools

Through the development of standards and piloting of e-assessment related technologies

To provide guidance for institutions on effective practice in this area

E-Assessment JISC Work Done to Date a wide range of tools and software case studies exploratory reports a reference model a glossary and a roadmap Projects: REAP reengineering assessment practice FREMA framework reference model for assessment IBIS Item Bank Infrastructure Study APIS assessment provision through interoperable segments ASAP automated system for the assessment of programming CATS constructing assessments using tools and services, ISIS, Open Mentor, PyAssess, Web PA…..

a wide range of tools and software

case studies

exploratory reports

a reference model

a glossary and a roadmap

Projects: REAP reengineering assessment practice FREMA framework reference model for assessment IBIS Item Bank Infrastructure Study APIS assessment provision through interoperable segments ASAP automated system for the assessment of programming CATS constructing assessments using tools and services, ISIS, Open Mentor, PyAssess, Web PA…..

e-Assessment JISC Work Underway AQURATE Feb 07 – March 08 QTI2 authoring tool (Kingston University) Minibix Feb 07 – March 08 QTI2 Item Bank (Cambridge University) ASDEL Feb 07 – March 08 Assessment delivery engine for QTI2 questions (Southampton University) MCQFM Jan 07 – Aug 07 Web Service for Rapid Creation of QTI2 questions (Hertfordshire University) XMARKS Jan 07 – Oct 07 Web Service for exposing assessment and marks data http://www.jisc.ac.uk/whatwedo/programmes/programme_capital/capital_funded/capital_elearning_0406funded.aspx

AQURATE Feb 07 – March 08 QTI2 authoring tool (Kingston University)

Minibix Feb 07 – March 08 QTI2 Item Bank (Cambridge University)

ASDEL Feb 07 – March 08 Assessment delivery engine for QTI2 questions (Southampton University)

MCQFM Jan 07 – Aug 07 Web Service for Rapid Creation of QTI2 questions (Hertfordshire University)

XMARKS Jan 07 – Oct 07 Web Service for exposing assessment and marks data

http://www.jisc.ac.uk/whatwedo/programmes/programme_capital/capital_funded/capital_elearning_0406funded.aspx

e-Assessment JISC Priorities http://www.jisc.ac.uk/whatwedo/themes/elearning/programme_elearning.aspx Investigate discipline-specific use of assessment Automatic marking of free-text assessments Web 2.0 and pedagogy / innovation Further development of domain mapping Tangible benefits QTI – user needs; support tools development, validation, dialects etc Large scale demonstrators of existing toolkits Further toolkit development, test administration, accessibility, quality Standardised assessment data transfer Cross stakeholder and partner initiatives Item Banking Institutional InfoKit

http://www.jisc.ac.uk/whatwedo/themes/elearning/programme_elearning.aspx

Investigate discipline-specific use of assessment

Automatic marking of free-text assessments

Web 2.0 and pedagogy / innovation

Further development of domain mapping

Tangible benefits

QTI – user needs; support tools development, validation, dialects etc

Large scale demonstrators of existing toolkits

Further toolkit development, test administration, accessibility, quality

Standardised assessment data transfer

Cross stakeholder and partner initiatives

Item Banking

Institutional InfoKit

JISC e-Assessment How to Contribute [email_address] Assessment SIG coordinator & SIG Expert consultant Jiscmail lists (e-Assessment, CETIS-QTI-SIG, JISC-Announce) Assessment related events…. Open call Guide to bidding http://www.jisc.ac.uk/fundingopportunities/bidguide.aspx

[email_address]

Assessment SIG coordinator & SIG

Expert consultant

Jiscmail lists (e-Assessment, CETIS-QTI-SIG, JISC-Announce)

Assessment related events….

Open call

Guide to bidding http://www.jisc.ac.uk/fundingopportunities/bidguide.aspx

JISC Assessment Expert Consultant Expert Consultant ( details ) Support the JISC programme manager in: core set of documents for each activity area: Definition and scope Overview of recent and current JISC-funded activities Overview of other significant initiatives or developments in the sector Stakeholder map Summary of progress to date on description and modelling of the area, such as through reference models or domain maps. Baseline description of current sector practice and JISC-funded developments, with a gap analysis of potential areas for future work Development priorities for the activity area Providing feedback to projects on their plans and reports….

Expert Consultant ( details )

Support the JISC programme manager in:

core set of documents for each activity area:

Definition and scope

Overview of recent and current JISC-funded activities

Overview of other significant initiatives or developments in the sector

Stakeholder map

Summary of progress to date on description and modelling of the area, such as through reference models or domain maps.

Baseline description of current sector practice and JISC-funded developments, with a gap analysis of potential areas for future work

Development priorities for the activity area

Providing feedback to projects on their plans and reports….