The formulation of an ISDT for e-learning

The formulation of an ISDT for e-learning David Jones (CQU) Shirley Gregor (ANU) http://cq-pan.cqu.edu.au/david-jones/

The Question

How do you design and support an information system for e-learning within an institution of higher education?

First asked in 1996 Still an open question…..

It is important The importance of information and communication technologies and e-learning in promoting open, distance and flexible education in contemporary universities cannot be denied (deFreitas & Oliver, 2005) . Overall % of schools identifying online education as a critical long-term strategy grew from 49% in 2003 to 56% in 2005. (Allen & Seaman, 2005)

There are problems Vendor identified, best-practice institutions report around 50% faculty adoption (Sausner, 2005) Most HEIs are still struggling to engage a significant percentage of students and staff in e-learning (Salmon, 2005) a relatively stable minority of Chief Academic Officers believe that their faculty fully accept the value and legitimacy of online education (Allen & Seaman 2005)

There are problems … not particularly innovative, may limit the incentive to innovate, limited in quality and may restrict the ability to integrate with other systems (Alexander, 2001; Anonymous, 2004; Paulsen, 2002) … Web-based learning lacks a disciplined, systematic approach to the development process.….much of its construction is carried out without a true analysis of the requirements that are proper to Web-based learning (Hamid, 2002) .

It is difficult ..complex systems incorporating a variety of organisational, administrative, instructional and technological components (Avgeriou, Papasalouros, Retalis, Skordalakis, 2003) Absence of theory for e-Learning IS (Jones, Gregor and Lynch, 2003) Scholars in Information Systems can offer vision on structures and processes to effectively implement technology-mediated learning initiatives (Alavi & Leidner, 2001)

Proposed contribution

ISDT that provides 1 answer to the question

Greater adoption, customization, differentiation

Based on 10 year action research cycle involving "real" system (1200+ staff, 22,000+ students)

Demonstrates novel use of ISDT

To understand/improve EXISTING information system

What we did

Plan Act Reflect Watch 1996-2003 3 generations of action research 2002- Formulation of Information Systems Design Theory Webfuse 24 research publications 5 research publications

What we'll do today

Laid out the context 

Describe three generations

Why and what (The ISDT)

Instantiation (Webfuse) and results

Reflect

Value of ISDT applied to existing systems

What can be IS design science research?

Generation 1: 1996-1999 Build it and they will come

Why

1996 - the web is a "good thing"

Design

Single academic one teaching free term

Aim to support

Web-based learning and teaching

Faculty website

What - the ISDT

Kernel Theories Not explicitly stated DE/education literature & experience Hypermedia templates, s/w wrappers Flexibility & ease of use "Waterfall" Templates Possible to construct a usable system Will be more flexible Design Method Meta- requirements Meta-design Testable hypotheses

Meta- requirements Kernel Theories Not explicitly stated DE/education literature & experience Hypermedia templates, s/w wrappers "Waterfall" Templates Possible to construct a usable system Will be more flexible Design Method Meta-design Testable hypotheses What's needed? Flexibility & ease of use

Meta-requirements

flexibility and the ability to adapt to change

platform independence

use of Internet and other widely accepted standards

not reinventing the wheel

maximising the choice and flexibility provided to students and teachers

providing the tools not the rules

minimising online time

minimising the new skills required to use the system

being freely available

(Jones & Buchanan, 1996; McCormack & Jones, 1997)

Support for variety and change

No specific functional requirements

No support for specific learning theory

Basic support for

Information distribution

Communication

Assessment

Management

Design (McCormack & Jones, 1996; Hansen and Frick, 1997)

Flexibility & ease of use Meta- requirements Kernel Theories Not explicitly stated DE/education literature & experience Hypermedia templates, s/w wrappers Flexibility & ease-of-use "Waterfall" Templates Possible to construct a usable system Will be more flexible Design Method Meta-design Testable hypotheses Product

Each course website starts with a default skeleton of web pages There can be different skeletons

Content + conversion + appearance

Edit link

Templates as wrappers Users Discussion forum Chat room Online quizzes

Flexibility & ease of use Meta- requirements Kernel Theories Not explicitly stated DE/education literature & experience Hypermedia templates, s/w wrappers "Waterfall" Templates Possible to construct a usable system Will be more flexible Design Method Meta-design Testable hypotheses Process

Flexibility & ease of use Meta- requirements Kernel Theories Not explicitly stated DE/education literature & experience Hypermedia templates, s/w wrappers "Waterfall" Templates Possible to construct a usable system Will be more flexible Design Method Meta-design Testable hypotheses Expected outcomes

Generation 1 Instantiation

Implementation

Technology

Perl, Apache

CGI scripts

People

On-going support Webmaster other general staff

New developments: mostly 1 academic + project students

Results

150+ course websites,

With 100,000+ pages

Maintained by almost 100 staff

Only 5 of 60 full-time academic staff making significant use of e-learning

Usage dominated by system developer

83% of content for 1 term 1998 from 1 course

Generation 2: 1999- Give them a reason to come

Why

Product focus and process ignorance

Problems

Limited use

Not enough evolution

Need to fix the process

Greater adoption

Greater evolution

What - the ISDT v2.0

Flexibility & ease of use Meta- requirements Diffusion theory, design patterns Adopter focus, pattern mining Possible to construct a usable system Will be more flexible More acceptable = greater adoption Design Method Testable hypotheses Kernel Theories DE/education literature & experience Hypermedia templates, s/w wrappers Templates Meta-design

Generation 2 - Instantation

Resources

No significant addition of development resources

What to implement?

Usually, driven by

Technology

Strategic planning

Pedagogy

We adopted diffusion theory

Maximise chance of adoption, minimise effort

Diffusion theory Rate of adoption Level of Reinvention Perceived attributes Innovation decision Communication Channels Social System Change Agent Efforts (Jones, Jamieson & Clark,2003) Relative advantage Compatibility Complexity Trialability Observability Optional Collective Authority Mass-media or interpersonal Cosmopolite or local Social structure Communication structure System norms Client contact: effort, timing. Rapport, empathy, credibility and homophily

Diffusion theory Rate of adoption Level of Reinvention Perceived attributes Innovation decision Communication Channels Social System Change Agent Efforts

"Pattern" Mining Academic Staff Template Repository The expected The unexpected

"Pattern" Mining v2.0 Academic Staff Template Repository The expected The unexpected

"Pattern" Mining v2.0 Academic Staff Template Repository The unexpected The expected

Generation 2 - Results

Results

Some improvements

But resources not appropriate

Limited ability to adapt

… more covered in generation 3

Generation 3: 2001- Build what they want, faster

Why

Generation 2 limited by

Inadequate resources

Poor technology

Poor process

Context changes

2000 new Dean

1999-2001 - 33% growth in students

What - the ISDT v3.0

Flexibility & ease of use Meta- requirements .. Agile, emergent, ateleological ..XP, emergent development Possible to construct a usable system Will be more flexible More acceptable = greater adoption Design Method Testable hypotheses Kernel Theories DE/education literature & experience … .OO design patterns … design patterns Meta-design

Generation 3 - Instantiation

Product and Process

Development team

1 lead developer

3-5 other developers

Patterns (e.g.)

Model/View/Controller web development

Greater use of factory pattern

Extreme Programming aspects

Test driven development/Unit tests

Public User Stories

Stand-up meetings

Generation 2 - Results

Customisation The key to easy use of OASIS is that it is not a off the shelf product that is sooooo (sic) generic that it has lost its way as a course delivery tool.

Building trust ..the precedent of other IT systems made available in Infocom (…) suggests that it would be extremely user friendly for people with very limited computer competence/confidence. my positive experience with other Infocom systems gives me confidence that OASIS would be no different. The systems team have a very good track record that inspires confidence

Ease-of-use I identify myself as coming from the technophobic extreme of the faculties (I have an Arts background) but even I find the systems the team have set out relatively easy to use

OASIS - ease of use/usefulness Non-users: 19% (n=22) Users: 37% (n=99)

Number of Staff Users Generation 1 Generation 2 Generation 3

Number of Staff Users Generation 1 Generation 3 Generation 2 Shadowy and feral lead to possible system shutdown

Number of Staff Users Generation 1 Generation 3 Generation 2 System accepted at organisational level

Student Portal

Staff Portal - # users

Staff Portal - # requests (* until Feb 18, 2006)

Online assignments

Implications for design science research and IS

Usefulness of ISDTs