QTI 2.1

Overview Approaches to implementation- an update Korean QTI 2.1 tools: Daulsoft teaching mate Hangul word processor Learning standard validator QTI for Math profiling IMS implementation survey Strategy for QTI Math profile: Basic – Medium – Large QTI + Math extensions Expand QTI 2.1 with Math > profile down for UK Merge or diverge from UPMC Math profile?

Approaches to implementation- an update

Korean QTI 2.1 tools:

Daulsoft teaching mate

Hangul word processor

Learning standard validator

QTI for Math profiling

IMS implementation survey

Strategy for QTI Math profile:

Basic – Medium – Large QTI + Math extensions

Expand QTI 2.1 with Math > profile down for UK

Merge or diverge from UPMC Math profile?

Assessment system infrastructure

Use case 1: subject centre Division: Subject centre bank and authoring Institutional learning system 3d party service delivery Tight interoperability points internal, loose interoperability points external Resources spread to those who care most Strong reliance on QTI

Division:

Subject centre bank and authoring

Institutional learning system

3d party service delivery

Tight interoperability points internal, loose interoperability points external

Resources spread to those who care most

Strong reliance on QTI

Use case 2: regional federation Division: Each institution owns their own VLE and tests Federation owns everything else Easy integration Adequate authoring tools a challenge Does not rely much on QTI

Division:

Each institution owns their own VLE and tests

Federation owns everything else

Easy integration

Adequate authoring tools a challenge

Does not rely much on QTI

Use case 3: national resource centre Division: Centre owns the item bank and contents Institution everything else (other things being equal) Complex many-to-one coordination points Adequate authoring tools a challenge Heavy reliance on QTI

Division:

Centre owns the item bank and contents

Institution everything else (other things being equal)

Complex many-to-one coordination points

Adequate authoring tools a challenge

Heavy reliance on QTI

Use case 4: Assessment content publishers Division National centre contracts content and tool vendor, holds item bank Content vendor authors content, holds copyright Tool vendor sells test tools Institution does rest Relatively many external interoperability points Relies heavily on QTI if sustainability is a criterion

Division

National centre contracts content and tool vendor, holds item bank

Content vendor authors content, holds copyright

Tool vendor sells test tools

Institution does rest

Relatively many external interoperability points

Relies heavily on QTI if sustainability is a criterion

Use case 5: Institutional distributed learning environment Division: Institution creates and ownes all content Authoring service vendor provides a range of tools and a storage facility Test service provider provides test composition and delivery Small number of external interoperability points Medium reliance on QTI

Division:

Institution creates and ownes all content

Authoring service vendor provides a range of tools and a storage facility

Test service provider provides test composition and delivery

Small number of external interoperability points

Medium reliance on QTI

QTI assessment system infrastructure Therefore, for greatest interoperability: Inverse relation between the complexity of the data exchanged, and the variation in applications that process that data Hand responsibility for component to party with greatest interest For profiling this means Subjects set requirements for rich profile (assuming compromise or centralised infrastructure) Else: lowest common technical denominator profile

Therefore, for greatest interoperability:

Inverse relation between the complexity of the data exchanged, and the variation in applications that process that data

Hand responsibility for component to party with greatest interest

For profiling this means

Subjects set requirements for rich profile (assuming compromise or centralised infrastructure)

Else: lowest common technical denominator profile

The role QTI plays in the infrastructure QTI as exchange format across the system + Consistent semantics - Difficult profile coordination problem between systems and over time QTI as intermediary format between systems + Supports legacy systems now - Semantic roadblocks (unacceptable degradation between authoring and use) For profiling, this means: Intermediary format suits lowest common technical denominator profile Exchange format suits rich subject profiles

QTI as exchange format across the system

+ Consistent semantics

- Difficult profile coordination problem between systems and over time

QTI as intermediary format between systems

+ Supports legacy systems now

- Semantic roadblocks (unacceptable degradation between authoring and use)

For profiling, this means:

Intermediary format suits lowest common technical denominator profile

Exchange format suits rich subject profiles

The economics of interoperability The expensive part:

The expensive part:

Balancing demand with capability

The combinatorial interoperability problem Symmetrical, many-to-many interoperability; 8 systems, 56 connections that need to work

Symmetrical, many-to-many interoperability; 8 systems, 56 connections that need to work

The combinatorial interoperability problem ctd. Asymmetrical, many-to-many interoperability; 8 source systems, 2 consuming systems, 16 connections that need to work This how JPG, BIND, the web etc. work

Asymmetrical, many-to-many interoperability; 8 source systems, 2 consuming systems, 16 connections that need to work

This how JPG, BIND, the web etc. work

Consequences for assessment To align QTI capabilities with demand, and ensure interoperability: Many authoring tools Many test composition tools Generic L/CMSs for item banks A couple of assessment delivery engines

To align QTI capabilities with demand, and ensure interoperability:

Many authoring tools

Many test composition tools

Generic L/CMSs for item banks

A couple of assessment delivery engines

Tool architecture Engines can be included in multiple ways: Library / engine (e.g. qti engine) Plug-in (e.g. Playr Moodle plug-in) Web service Widget

Engines can be included in multiple ways:

Library / engine (e.g. qti engine)

Plug-in (e.g. Playr Moodle plug-in)

Web service

Widget

Thank you! Wilbert Kraan [email_address]

Wilbert Kraan

[email_address]