Long-Term Retention and Reuse of E-Learning Objects and Materials

Long-Term Retention and
Reuse of E-Learning Objects
and Materials
Report Commissioned by the Joint
Information Systems Committee (JISC)
Version 1.4, November 11th
, 2004
Ed Barker, Institute for Computer Based Learning
Hamish James, Arts and Humanities Data Service
Gareth Knight, Arts and Humanities Data Service
Colin Milligan, Institute for Computer Based Learning
Malcolm Polfreman, Arts and Humanities Data Service
Roger Rist, Institute for Computer Based Learning

Long Term Retention and Reuse of E-Learning Objects and Materials
Institute for Computer Based Learning (ICBL)
The ICBL is part of the School of Mathematics
and Computer Science at Heriot-Watt
University. Established in 1990, the ICBL has
been actively studying design, development
and evaluation issues in the use of learning
technology in education.
Arts and Humanities Data Service (AHDS)
The AHDS is a UK national service funded by
the Joint Information Systems Committee and
the Arts and Humanities Research Board.
Organised via an Executive at King's College
London, and five Centres at various Higher
Education institutions, the AHDS aids the
discovery, creation and preservation of digital
collections in the arts and humanities.
i

Contents
Contents.........................................................................................................................ii
Acknowledgements......................................................................................................iii
1 Executive Summary....................................................................................................1
2 Recommendations.......................................................................................................2
3 Introduction.................................................................................................................5
4 Views on the Long-Term Value of E-Learning Objects and Materials......................7
5 Designing for Reuse..................................................................................................12
6 Encouraging Reuse of E-learning Materials.............................................................22
7 Infrastructure for Retaining E-learning Materials and Objects.................................28
8 Digital Content in E-learning....................................................................................41
9 References.................................................................................................................53
Appendix A: Information on Projects.........................................................................66
Appendix B: Interview List for Long Term Retention Project...................................68
Appendix C: Focus Group...........................................................................................69
Focus Group Write-Up................................................................................................69
Appendix D: Analysis of Interviews, Focus Groups and JORUM study....................83
ii

Acknowledgements
The ICBL and AHDS acknowledge with gratitude the following people and
organisations
• Michael Breaks, University Librarian, Heriot Watt University
• Joe Wilson, Scottish Further Education Unit (now Business Manager,
SHN/SVQ Business and IT)
• Phil Barker, CETIS Metadata and Digital Repositories SIG Co-ordinator
• Andy Powell, Distributed Systems, UKOLN and JISC Resource Discovery
Network (RDN)
• Kathy Wiles, Learning and Teaching Support Network Generic Centre (now
Director, Centre for Academic Development, University of Newcastle)
• Intrallect, including Charles Duncan, CEO and Peter Douglas, Consultancy
Director.
• Sarah Porter, Learning and Teaching Programme Director, JISC
• Steve Bailey, Records Manger, JISC
• Balviar Notay, Programme Manager, JISC
• Moira Massey, Jackie Carter, Sarah McConnell, JORUM
• Howard Noble, Educational Interoperability Specialist, Learning Technology
Group, Oxford University
• David Prescod, Acting Head, NLN materials Team
• Jean Ritchie, Media and Learning Technology Services, Edinburgh University
• Mary MacDonald, COLEG Manager
• Graeme Turnbull, Publishing Manager, SCRAN
• Andrew Comrie, Assistant Principal, Lauder College
• Derek Barker, Reader, Napier University
In addition, the study team would like to thank all those who participated in the focus
group and those who agreed to be interviewed for this study.
iii

1 Executive Summary
This study has been commissioned by JISC to examine the challenges which will
affect the retention and re-use of learning objects and materials, and to make
recommendations to JISC on how best to develop its e-learning and digital
preservation activities to address these challenges. The focus is on the creation and re-
use of e-learning objects, the interoperability issues within current learning systems,
and the requirements for storage of materials in digital repositories. The findings and
recommendations of this study are intended to offer JISC and institutions assistance in
their considerations of learning objects and materials as a new collecting area.
The idea of readily available collections of high quality, reusable learning objects has
caused much expectation in the e-learning field. However there are problems with
curriculum change and staff involvement and the need is to develop and manage
learning materials in such a way that it will be possible to modify and re-use the
materials in an efficient and effective way. The management of teaching and learning
materials for long term use has been identified as an area in which there is room for
improvement. Good quality resources have not always been utilised to their full
potential and there is often duplication of work.
Care must be taken to make sure that pedagogical and practical issues are considered
in order to make appropriate use of the technology. There is a need to develop
methodologies and models for the institutional utilisation of reusable learning
materials.
By creating and managing resources in a sensible manner, good materials will be
easily retrieved and effectively used. As well as having the potential to reduce effort,
the resources may become important assets, which could be shared with or sold to
other institutions. However, the whole methodology is new and unproven so
institutions should understand that the risks are high.
In the future more attention will have to be paid to making sure resources produced
using national funding are designed in such a way that they can be used by the target
audience for the required period of time. This may involve conditions of contract
about obtaining licences for third party materials, use of software and file types. It
could also be mandatory to deposit the materials in a safe long term storage facility.
Selected past projects could be repurposed to improve their chances of reuse and
placed within the storage facility.
Methods of describing resources need to be looked at in detail, as this will affect how
they can be used. It will also be necessary to keep the resources along with
information about licences and evaluation. This information will have to be linked
together in some way and methods of updating the resources and related information
will have to be considered. This problem will become much harder if resources are
transferred from collection to collection or if links are used. The uptake of standards
can help to alleviate some of these problems.
1

2 Recommendations
2.1 List of Recommendations
The report contains the following recommendations.
Recommendation 1
JISC should ensure that all e-learning materials created under JISC funded
programmes are available without restriction to UK HE and FE.
Recommendation 2
The failure of many e-learning initiatives to take root in higher and further education
is a significant concern. JISC should investigate the causes of this failure further.
Future funding for e-learning initiatives should include a requirement for projects to
include an evaluation of usage so that problems can be more readily identified and
lessons applied to subsequent activities.
In-depth comparative studies of successful and unsuccessful e-learning resources
should be commissioned.
Recommendation 3
Work has been done by the JISC Online Repository for [Learning and Teaching]
Materials (JORUM) study (JORUM, 2004e) into methods that the end user could use
to obtain information about the quality of a learning object. Further research is now
needed into methods for measuring the quality of learning objects.
Recommendation 4
A model licence for sharing e-learning material should be developed.
• Particular attention should be paid to protecting the moral rights of the
original creator.
• Licence terms should ensure that standard digital preservation strategies, such
as file format migration, are not precluded.
• Data protection and copyright issues over user feedback (potentially a
valuable source of quality assessment information) need to be addressed.
Recommendation 2
Guidelines for creating reusable e-learning materials have been developed by the
National Learning Network (NLN), Paving the Way document (NLN, n.d.), Ferl and
Exchange for Learning Programme (X4L) Healthier Nation (X4L, n.d.). Work should
be undertaken to determine if these and other guidelines can be amalgamated into a
single best practice guide for UK HE and FE.
Recommendation 6
2

JISC should fund a small survey study to determine which existing e-learning
materials are of sufficient value to invest in making them standards-compliant. The
study should focus on the costs and benefits of updating materials as compared with
maintaining them in their present form.
Recommendation 7
Responsibility for the creation, delivery and long-term management of e-learning
materials is often fragmented and unclear. Institutions should develop e-learning
material management policies to clarify roles and responsibilities.
Recommendation 8
Guidance on quality assurance procedures for e-learning material metadata should be
developed.
Recommendation 9
Currently it is often difficult to quantify the cost of collection management because
there are a lot of people and processes involved. Institutions could track costs of
resources management in a more coordinated manner. JISC should seek to fund a
comparative study investigating current collections projects such as SCRAN, HLSI
and the Resource Discovery Network (RDN) hubs in order to obtain information
about costs associated with managing e-learning materials.
Recommendation 10
E-learning repositories should establish practical definitions of the significant
properties of different types of e-learning content. These might be expressed through
restrictions on acceptable file formats, guidance on how to use particular formats, and
requirements for particular metadata to accompany content.
Recommendation 11
E-learning repositories should consider limiting the range of file formats they will
accept within learning objects or other types of e-learning materials.
Recommendation 12
A number of existing advisory and training services are relevant to the creation of
high quality e-learning material within institutions. Relevant services include
Netskills, the newly established Digital Curation Centre (DCC), the Arts and
Humanities Data Service (AHDS), the Technical Advisory Service (TASI) for
Images, the Centre for Educational Technology Interoperability Standards (CETIS),
and the Managing Agent and Advisory Service (MAAS). JISC should consider ways
of targeting promotion of these services at e-learning material creators.
Recommendation 13
The development of e-learning materials to appropriate standards and specifications
will greatly ease preservation issues associated with long-term retention, and will
simplify reuse.
3

Institutions and JISC should work together to develop a training scheme for tutors and
lecturers interested in developing e-learning materials. The online nature of most e-
learning material suggests that a formal relationship with the Netskills service may be
worthwhile.
Recommendation 14
Institutions should seek to apply any digital preservation work being undertaken in a
library or archival setting with e-learning systems and content development that may
also be occurring.
4

3 Introduction
3.1 Background to the Study
There is growing recognition that Information and Communications Technology
(ICT) can play an important role in support of learning and teaching. While e-learning
is still in an early phase of its evolution, investment in e-learning is already
substantial. UK Higher and Further Education institutions have directed considerable
effort and resources towards creating and acquiring e-learning content. To secure the
long-term future of this investment, effort must now be put into ensuring that e-
learning content can be retained in a usable state as long as it is pedagogically
relevant.
This study has been commissioned by the Joint Information Systems Committee
(JISC) in order to examine the factors that may affect the Long-Term Retention and
Reuse of E-learning Objects and Materials. The study focuses on the requirements for
the long-term retention and reuse of e-learning resources in the context of the JISC
Information Environment (IE). For the purposes of this study, long-term can be
defined as the period of time during which the hardware, software, formats and
standards used to create and access digital objects, such as e-learning resources,
become obsolete.
JISC has played a significant role in advancing both the e-learning agenda and the
digital preservation agenda in the UK since 1995. This study forms part of the
implementation of JISC’s Continuing Access and Digital Preservation Strategy 2002-
5 (JISC, N.D.(c)), and supports JISC’s e-learning programmes.
The JISC’s on-going commitment to the e-learning agenda is to develop a coherent
vision that can connect different systems and support a diverse range of pedagogic
models and types of institution. The JISC e-learning programme for 2003-06
incorporates three strands: e-learning and pedagogy; technical frameworks for e-
learning and innovative technologies.
The JISC Continuing Access and Digital Preservation Strategy 2002-5 sets out JISC’s
commitment to developing the UK digital preservation agenda. One of the key
initiatives in the implementation plan for the JISC Continuing Access and Digital
Preservation Strategy 2002-5 is the completion of preservation risk and retention
criteria assessments for digital content, which in turn will inform and prioritise the
development of future services and calls in digital preservation. Alongside earlier
studies on web archiving, e-prints and e-journals, this study contributes to that aim.
3.2 Methodology
The study was conducted over the period July 2003 to March 2004 by the Institute for
Computer Based Learning (ICBL) at Heriot-Watt University, assisted by the Arts and
Humanities Data Service (AHDS).
Information was obtained through desktop studies, a focus group (Appendix C),
telephone interviews and face-to-face interviews (Appendix B). Information was also
5

obtained from the recently published JORUM (JORUM, 2004c & 2004d), and this
study makes use of material drawn from telephone interviews, focus groups and a
questionnaire conducted by the JORUM study team. Findings from these various
sources are summarised in Appendix D.
Resource creators and publishers were consulted from a range of projects including:
Scottish Cultural Resources Access Network (SCRAN), COLEG, European
Meteorological Education and Training (EuroMET) and the National Learning
Network (NLN). Several of these projects have existed for a long period of time and
the reasons for their longevity and the problems they have had to deal with provided
important background information for the study. Further information about these and
other projects referred to in this study are given in Appendix A.
Interviews with an assistant principal, a university librarian and the Scottish Further
Education Unit (SFEU) were used to give a perspective on institutional e-learning
strategy. Representatives from JISC were also interviewed to obtain information
about current plans and directions for resource management in FE and HE.
FE and HE lecturers were invited to the focus group. The focus group participants can
be considered to be individuals who have a strong interest in e-learning.
Apart from those already mentioned, there are many other groups who have a stake in
the long-term retention and reuse of e-learning objects and materials in HE and FE,
most obviously students. However, given the scope of this study, it was not
considered feasible to interview a representative sample of students.
Comments on this report may be directed to:
Dr Roger Rist
Institute for Computer Based Learning
Heriot-Watt University
Edinburgh
EH14 4AS
roger@icbl.hw.ac.uk
6

4 Views on the Long-Term Value of E-Learning
Objects and Materials
Since the 1990s there has been a persistent effort, pursued through an array of
initiatives, to promote learning technology in UK Higher and Further Education (HE
and FE). Major past activities include the Information Technology Training Initiative
(ITTI) and the Teaching & Learning Technology Programme (TLTP). Other
activities, such as the Fund for the Development of Teaching and Learning (FDTL)
continue. JISC programmes, such as the JISC Technology Applications Programme
(JTAP) and the 5/99 Learning and Teaching Programme have sought to investigate
and develop the technological framework needed to manage and deliver e-learning
content.
Mention must be made also of the NLN, which is one of the most significant
contributions to e-learning in the UK and is “designed to increase the uptake of
Information and Learning Technology … across the learning and skills sector in
England” (NLN, N.D.).
Like many new concepts, e-learning has been oversold on occasion and this has
created unrealistic expectations about how quickly and dramatically traditional
approaches to learning might be transformed. The failure to live up to earlier
hyperbole around e-learning has damaged the credibility of e-learning developments
within much of the academic community, as a number of respondents (focus group
and interviews) to this study observed:
• “Hype is often necessary to get funding for e-learning projects”
• “Digital resources can mutate and lose provenance quickly”
• “E-learning objects have fuzzy boundaries unlike paper resources”
• “Communities can not agree on definitions for e-learning concepts”
• “At the moment learning object systems have taken bad learning practices and
solidified – do not encourage group work.”
In general, the focus groups and interviews conducted for this study suggest that there
is still “far greater confidence in books than [in e-]learning materials”. However,
despite this view, respondents to both this study and to the JORUM scoping study
also noted that there is a trend towards greater use of digital resources in learning and
teaching:
• “Just about every FE college in Scotland has a VLE strategy”
• “There is a move towards VLEs in Universities”
• “E-Journals are being used more”
Deliberate efforts have been made to ensure that the technological innovations of e-
learning are embedded into wider professional practice. In the second phase of the
Computers in Teaching Initiative (CTI), for example, 24 subject specific centres were
established "to maintain and enhance the quality of learning and increase the
effectiveness of teaching through the application of appropriate learning technologies"
(Martin, 1996). In 2000, the CTI Centres were replaced by the Learning and Teaching
7

Subject Network (LTSN), which was given a wider remit to promote “high quality
learning and teaching in Higher Education through the development and transfer of
good practices in all disciplines” (LTSN, 2002). The LTSN centres are now in turn
being incorporated into the new Higher Education Academy, which will consider the
entire student experience (Higher Education Academy, N.D.) continuing a process of
integrating e-learning with learning in general.
In Further Education, BECTA (British Educational Communications and Technology
Agency), and its forerunner the National Council for Educational Technology
(NCET), have undertaken a similar role, offering services such as FERL, an
information service covering learning technologies, learning management
technologies and approaches to teaching.
Unfortunately, despite these efforts, much of the e-learning content created to date has
failed to attract a sizable and enduring audience. Lack of promotion and awareness
(Ritchie, 2002), and the variable quality of resources have affected the uptake of
many e-learning offerings. This has lead to a situation where the creation of new e-
learning materials has continued while existing materials have proved difficult to
sustain. Respondents noted that e-learning materials are sometimes promoted before
they are fully developed and ready for use, but that even when products are of high
quality, it is not always easy to get them used in practice:
• “Even if learning object systems are good people will not use them”
• “Dissemination is not that easy – communication between staff is not always
good. Different dissemination mechanisms are necessary in different
institutions”
• “The real problem is letting people know and keeping awareness up”
• “Need to promote e-learning materials to broaden usage keeping in mind
maintenance and ongoing support”
• “Responsibility for resource awareness is a big problem”
Responses to the JORUM questionnaire (JORUM, 2004c) highlighted similar
concerns:
• “E-Learning projects only used by small subset of the community“
• “I am a library systems manager and I can see little demand for this kind of
thing. Even E-Journals get very little use and a lukewarm reception from even
the most enthusiastic on-line individuals despite years of trying to promote
such products. I find it easier to help people find books and journals that they
can pick up and read with no barriers, password, inductions etc”
Some respondents made comments highlighting the role short-term funding may play
in discouraging the reuse of e-learning materials:
• “opportunistic projects are not planned to be sustainable”
• “No-one is responsible for a project once it has concluded”
Perhaps the most telling comment was the simple observation that “short term funding
implies long term uncertainty”. The Use of Metropolitan Area Networks Initiative
8

(UMI) serves as a cautionary example. The initiative funded 19 projects and ended in
1998. Of those 19 projects, only four project websites are still available. Following
the UMI programme, SHEFC (Scottish Higher Education Funding Council) funded
19 projects under the ScotCIT (C & IT Programme of the Scottish Higher Education
Funding Council) programme. Nearly all the websites for these projects are still
available, but most do not appear to have been developed, or actively maintained
since funding for the projects ended.
As one respondent said: “There is some urgency here. As well as funding new
projects it is important to keep old ones [we] need to bring attention to the fact that
resources are being lost”. One learning technologist thought that saving resources
from past projects could “help the e-learning community build on resources rather
than the stop, start cycle which has been happening”.
Nevertheless, one of the conclusions drawn from the focus groups held by the
JORUM project was that “archiving of materials is a low priority for the community
and more research into digital preservation is necessary before this should be
considered” (JORUM, 2004c, pg 31). Interviewees for this study made similar
comments, noting the low awareness of the issues involved in keeping resources over
a long period.
Overseeing ongoing activities such as rights management, version control and
metadata maintenance could involve considerable costs. As respondents to the
JORUM scoping study observed, “no one wants responsibility for storage”; “who are
guardians, who takes charge?” Doubts were voiced by some respondents to the
JORUM scoping study and this study about the ability of institutions to take on the
management of e-learning materials:
• “Institutions lack a well defined structure to support the use of E-Learning
Materials”
• “Some institutions have a lot better attitude towards resource management
than others”
Regardless of these problems, there are issues that may well force institutions into
establishing means of retaining e-learning materials and objects. The record retention
schedules developed for higher education institutions by JISC (Parker, 2003) provide
a detailed overview of recommended practice for the retention of institutional records,
including those related to the preparation, delivery and assessment of teaching. The
guidance most directly relevant to e-learning materials is reproduced in Table 1. In
particular, there is a need to:
• Keep final versions of taught course materials available for the life of the
course and review for archival purpose
• Keep final versions of taught course assessments for the duration of the course
and review for archival purpose
• Keep taught course assessments completed by the student for the current
academic year + 1 year
9

Description Retention Period
Taught Course Preparation & Delivery
Records documenting the development of taught course
materials.
Life of course
Final versions of taught course materials. Life of course
Working papers documenting the planning and conduct
of teaching events.
Current academic year + 1
year
Taught Course Review
Records containing data on, and analyses of, student
numbers and other taught course statistics.
years
Records documenting routine solicited feedback on
taught courses from staff and examiners: individual
feedback.
years OR Life of course +
1 year
Records documenting routine solicited feedback on
taught courses from students: individual feedback.
Completion of analysis of
feedback
Records containing (anonymised) summaries and
analyses of routine solicited feedback on taught courses
from staff, examiners and students.
years OR Life of course +
1 year
Records containing reports of routine internal reviews of
taught courses
years
Records documenting the conduct and results of formal
reviews of taught courses, and the responses to the
results.
years
Taught Course Assessment
Records documenting the development of taught course
assessments.
Life of course
Final versions of taught course assessments. Life of course
Taught course students' submitted/completed
assessments.
year
Records documenting marks awarded to
submitted/completed assessments, including reviews in
response to notifications of mitigating circumstances
and academic appeals.
years
Records documenting awards and classifications. Current academic year + 6
years
Table 1: JISC Record Retention Guidelines
Most of the record types listed in table 1 would not require the retention of the e-
learning materials themselves, although complex interactive learning resources might
raise some issues with regard to retention of student assessments and feedback. The
simplest approach would be to ensure that this information can be extracted from the
interactive resource and stored separately, otherwise the entire interactive resource
might have to be maintained for five years or longer – time enough for technical
obsolescence to become a potential problem.
10

When e-learning material, no matter how good, fails to secure a foothold in the
learning environment it is most likely that the resource will ultimately be forgotten
and eventually lost. Any discussion of the long-term retention and reuse of e-learning
materials must recognise the ambivalence many in the academic community have
about the role and value of e-learning, and the affect this may have on the long-term
survival of e-learning materials. Care must be taken to ensure that e-learning materials
are promoted and perceived throughout the academic community as useful, sensible
and sustainable.
Recommendation 1
JISC should ensure that all e-learning materials created under JISC funded
programmes are available without restriction to UK HE and FE.
Recommendation 2
The failure of many e-learning initiatives to take root in higher and further education
is a significant concern. JISC should investigate the causes of this failure further.
Future funding for e-learning initiatives should include a requirement for projects to
include an evaluation of usage so that problems can be more readily identified and
lessons applied to subsequent activities.
In-depth comparative studies of successful and unsuccessful e-learning resources
should be commissioned.
Recommendation 3
Work has been done by the JORUM study (JORUM, 2004e) into methods that the end
user could use to obtain information about the quality of a learning object. Further
research is now needed into methods for measuring the quality of learning objects.
11

5 Designing for Reuse
5.1 Preliminaries: What is a Learning Object?
While (or probably, because) the concept of the learning object has become a locus of
argument about the future direction of e-learning, the exact definition of a learning
object is still a matter of debate (Ip, Morrison and Currie, 2001; Friesen, 2003;
Polsani, 2003). Wiley (2000, sec.1.1) defines a learning object as “any digital
resource that can be reused to support learning”. In The IEEE Learning Object
Metadata Final 1484.1.2.1 LOM draft standard document, similar thinking is
apparent in the definition of a learning object as “any entity – digital or non-digital –
that may be used for learning, education or training” (LTSC, 2002). But, without
qualification, these definitions are very broad, and come close to saying that a
learning object is synonymous with any digital resource, whatever its design or
original purpose. Indeed, potentially almost any type of resource can be of use for
learning. The National Learning Infrastructure Initiative (NLII), a U.S. organisation,
is typical with its expansive description of learning objects as:
digital resources, modular in nature that are use to support learning.
They include but are not limited to: simulations, electronic calculators,
animations, tutorials, text entries, Web sites, bibliographies, audio and
video clips, quizzes, photographs, illustrations, diagrams, graphs,
maps, charts, and assessments. They vary in size, scope, and level of
granularity ranging from a small chunk of instruction to a series of
resources combined to provide a more complex learning experience.
(NLII, 2003)
A more empirically helpful definition is provided by Dalziel:
A Learning Object is an aggregation of one or more digital assets,
incorporating meta-data, which represent an educationally meaningful
stand-alone unit.
(Dalziel, 2002, p.7)
This definition suggests that a resource cannot be regarded as a learning object until it
usefulness for learning is made explicit through the addition of metadata. The
JORUM project has adopted a similar definition, couched in slightly looser terms:
A learning object is any resource that can be used to facilitate learning
and teaching and has been described using metadata.
(JORUM, 2004, p.8)
This is the definition adopted in this report, with the proviso that this report only deals
with learning objects comprised of digital resources (some definitions allow for
learning objects to non-digital). The wider term e-learning materials is used to
describe both learning objects and other digital content that can be employed to aid
12

learning, but which lacks the additional connotations of accessibility, interoperability
and especially reusability that are typically associated with learning objects (Polsani,
2003).
5.2 Design Principles for Reusability
The definition of a learning object adopted in this report is intentionally broad. This is
done in an effort to avoid tying the report to any one view on how best to promote
reusability in e-learning. Many different approaches are currently being used to
develop e-learning materials for reuse, and they are dependent on the aims of the
creators and the audience they are designing for. For example, the MIT
OpenCourseWare project does not currently enforce a learning object type approach
for the teaching and learning content and material is made freely available on the
Internet. Conversely, the Curve project (CURVE, N.D.) at The Open University is
designed to be used internally and consists of well defined learning objects tagged
with IMS learning object metadata.
The range of different resource types that can be used in e-learning combined with
evolving techniques and standards for describing, storing and delivering e-learning
content are causing much confusion within the e-learning community. Worryingly, as
Wiley has noted:
The vast majority of existing digital educational resources cannot be
reused in current learning objects systems supposedly designed
specifically to support reusability.
Wiley (N.D.)
Maintaining any type of digital resource is, in the long-term, easier when due
attention has been given to relevant standards and good practices during the creation
of the resource (Jones & Beagrie, 2001). Retroactively improving digital resources so
that they can be deployed outside of the originally conceived context is a time
consuming and costly business. As Currier and Campbell observe:
In terms of developing reusable content, the major factor for success
that this part of DNER&LO [study] appears to highlight is the need for
planning at the start of an e-learning initiative for the considerable
amount of effort and expertise that must go into creating truly reusable
content.
(Currier and Campbell, 2002)
Elsewhere, Campbell (2003a) has identified the key factors that affect the reusability
of learning objects as granularity, technical dependency and content dependency.
These three factors all, in different ways, measure the sensitivity of a learning object
to the particular technical and pedagogical environment it is used in, and thus they in
large measure determine the potential for reusing e-learning objects in both the short
and long-terms.
13

5.2.1 Granularity
If a learning object is too large or conceptually complex it may be difficult to reuse it
in different contexts. Ensuring that learning objects are created at an appropriate level
of granularity is therefore seen as a key requirement for reusability. The appropriate
level of granularity may differ depending on factors such as object style, end users,
subjects covered and educational level, but, overall, the optimal level of granularity
can be found by considering two opposite tendencies. Larger blocks or chunks of
content will generally carry with them greater educational value, but at the same time
they will tend to be more integrated and context sensitive, making it difficult to reuse
the whole, or to extract parts of the material for reuse. Smaller chunks will be more
flexible, but will also, individually, be more limited in what they can provide to the
learner. Between the opposing trends of increasing flexibility and decreasing
educational value, Thorpe et al, (2003), suggests that an optimum granularity can be
found (Figure 1).
Figure 1: Adapted from Thorpe. M et al, 2003
At one extreme, several people who responded to the JORUM scoping study
questionnaire suggested that individual digital assets are more useful than learning
objects consisting of aggregated assets. The SCRAN collection of digital images for
arts and history, for example, is used by a large variety of end users including
Schools, HE and FE institutions (SCRAN, N.D.). This is partially because IPR issues
are less complex when dealing with individual assets, but also perhaps because
lecturers and teachers are already familiar with the idea of drawing upon collections
of assets (think of the traditional use of slide collections in teaching for example)
when constructing learning resources.
Librarians and archivist have long experience of managing collections of assets, and
although digital assets bring new challenges, they are more familiar than those posed
by learning objects.
Conversely, during the focus group for this study there was some discussion about e-
books, and how they might be used to hold e-learning materials. The perceived
advantage was that people are familiar with books and would be able to make value
judgements about the materials more easily. In addition, some content creators may
not want their material to be used out of context or if the materials are intended to be
used for income generation then with smaller chunks, it will be more difficult to
control usage. One interviewee suggested that it should be possible to achieve the best
of both worlds, essentially by creating e-books as a series of learning objects. This
would allow the end user to select smaller chunks of material for reuse, while at the
same time placing each chunk in a wider context. This concept has been used by the
HLSI repository, which contains objects at several layers of granularity (Barker &
Ryan, 2003)
Different granularities will be appropriate for different end user groups and analysis
of end user groups is necessary before decisions are made. In HE, for subjects that
have no defined curriculum, it may be appropriate to make use of digital assets to
provide more flexibility. For subjects which are more curriculum based such as
14

medicine and engineering, learning objects may be more appropriate. From the focus
groups and interviews, it appears that FE staff often have to produce “Just in Time”
courses and that they may make more use of larger Learning Objects such as objects
that have been made by the NLN.
5.2.2 Technical and Content Independence
When a learning object is technically dependent on other resources (for example,
HTML pages linked together into a fixed linear navigation sequence, or interactive
online content reliant on server side scripts for functionality), the settings in which it
can be reused may be severely limited. When the content held in a learning object
references other information external to the object (for example the previous or next
module in a course), reusability can again be limited.
Currently e-learning resources often rely on the user downloading software, external
links to other resources, and complicated web based interactions. There are several
resources created during the JISC 5/99 programme that cannot be easily deposited
into the JORUM repository because they were not designed to exist as independent
entities outside of the original technical environment used to run them (JORUM,
2003).
The consequences of adopting various typically used file types and software is
discussed in more depth in section 8.1. This choice can have an impact on how long
the resource can potentially be used for and the technical requirements for use. For
example, the EuroMET project, which started in 1995, has made use of Java. This has
given it a degree of continuing platform independence, helping the resource to survive
until today.
Similarly, if resources rely on other materials such as textbooks or websites then reuse
becomes less straightforward. Materials created by the Healthier Nation project (X4L,
N.D.(b)) encountered problems because the VLE resources contained links to
websites which could not be relied on.
There are long term projects such as the NEEDS resource collection (NEEDS, N.D.)
which makes use of a combination of external links and internally stored materials.
However, it seems that many of the long-term projects investigated for this study
(such as SCRAN, EuroMET and COLEG) store materials which have minimal
reliance on external resources.
It seems that additional information such as licence details and details of “use” may
add value. It would be possible to store all this information with the material in a
content package so that if the package is transferred, the end user would have their
own copy of all that is necessary to reuse the learning materials. However, some of
this information may change and the packages could potentially be passed from
system to system with the wrong information.
5.3 Metadata
One interviewee commented that separation of pedagogy, technology and structure of
e-learning materials will make them more reusable. The use of metadata, data about
data, can help with this process. Content packaging metadata can be used to hold an
15

aggregation of learning objects together with associated information such as digital
rights, descriptive metadata and evaluations.
5.3.1 Descriptive Metadata for Learning Objects
Locating relevant learning materials was identified as a problem by a number of
people contacted during this study. Both the classification and description of e-
learning materials are to some extent subjective and they require some anticipation of
the user’s needs by the person entering the metadata. Exactly what nature this
anticipation should take is unclear, and Mclean & Lynch (2003) suggest more
research is needed to discover:
what searches are really required – Google vs specialised search.
Needs to start with use cases so as to find out how people want to
discover learning objects and other assets that can be incorporated into
learning objects – people vs software agents, broad searches vs need
for precise searches to identify small components. More granular
models for Learning objects and “Lego” construction of metadata will
influence different search services.
Mclean & Lynch (2003)
Two standards for educational metadata are commonly used in the UK: Dublin Core
Educational (DC-Ed) and the IEEE-Learning Object Metadata Standard (IEEE LOM).
Both of these schemas can be used for educational material, although it has been
suggested by Olivier and Liber that one view which is beginning to emerge is that:
the ARIADNE/IMS/IEEE metadata set is best used for describing
resources that were explicitly created for learning purposes, while DC-
Ed is best used to describe more general content that can be used for
the purposes of learning.
Olivier & Liber (2003, p.150)
Dublin Core was not designed specifically for describing e-learning materials or
object management, but rather as a framework for shallow-level object description for
resource discovery across all domains. Nevertheless, the DC-Ed application profile,
developed recently by the DCMI Education Working Group, does go some way in
this direction. It introduces an <Audience> element with refinement for declaring the
intended audience and educational level of a resource – features that are importance
for all educational resources, not only those that are electronic. The <Mediator>
refinement allows the learning professional (e.g. “ESL teachers”) for whom the
resource is intended to be identified separately from the target end-users (e.g.
“elementary school students”; “deaf students”). The established (e.g. educational)
standard to which the resource conforms can be declared using the <Conforms To>
refinement of the <Relation> element. <Audience> is still somewhat unstable.
Controlled vocabularies are not yet finalised. Implementors are encouraged to develop
local lists of values, and to use them consistently. This is all a little crude. But DC
explicitly allows for the inclusion of elements, qualifiers and vocabularies from other
namespaces, such as IEEE-LOM that can offer more precise description.
16

DC-Ed and IEEE-LOM both share a resource discovery role but for the latter it is a
starting-point for more complex description of the structure of the learning object and
its management processes. It is more prescriptive than DC-Ed in relation to controlled
vocabularies, more of which are designed with e-learning material in mind. Key
aspects of e-learning materials addressed by IEEE-LOM include:
• Structure (collection, mixed, linear, hierarchical, networked, branched,
parcelled, atomic)
• Aggregation Level (the granularity of the resource)
• Life Cycle (draft, final, revised, etc.)
• Contributor role (graphical designer, technical implementer, content provider,
technical validator, educational validator, etc.)
• Technical requirements (such as operating system and browser) and
installation remarks
and a raft of detailed educational elements, including:
• Interactivity Type (Active, expositive, mixed, undefined)
• Learning Resource Type (Simulation, Questionnaire, Diagram, etc.)
• Interactivity Level (extent to which learner can influence the resource - very
low, medium, high, etc.)
• Semantic Density (conciseness of a resource - low, very high, etc.)
• Intended end user role (teacher, author, learner, manager)
• Typical Age Range
• Difficulty (very easy, difficult, etc.)
• Typical learning Time
• Purpose
Dublin Core and /IEEE LOM elements can be used together and their governing
bodies have reached a Memorandum of Understanding to co-ordinate their future
development.
The UKLOM CORE (previously UKCMF) is an application profile of the IEEE-LOM
that has been optimised for use within the context of UK education. Most comments
above about IEEE-LOM are therefore valid here. UK LOM CORE puts flesh onto the
IEEE-LOM structure – providing implementation guidelines, declaring which
elements are to be mandatory/optional and have minimum values. It recommends
vocabularies that are interoperable within a UK educational context (e.g. terms for
learning levels that make sense across the separate educational systems of the nations
of the UK). Much work still remains to be done in this area, though. JISC’s Exchange
for Learning Programme (X4L) strongly recommends the use of UKLOM CORE by
its projects.
As mentioned in section 2, quality assurance is important although it remains unclear
as to what the best approach is. It may be necessary to store information about quality
assurance in addition to LOM or DC-Ed metadata though this is unclear at present.
17

In addition information about how to use the resource could also be useful to the end
user. In particular, the IMS Learning Design specification is for describing resource
use in a pedagogically neutral manner so that lesson plans can be exchanged in an
accepted way. Although it is currently at an early stage of development, the JISC e-
learning and pedagogy programme is investigating its use.
More information about metadata issues is covered in (JORUM, 2004d)
5.3.2 Preservation Metadata
Preservation metadata … is the information necessary to maintain the
viability, renderability, and understandability of digital resources over
the long-term. Viability requires that the archived digital object’s bit
stream is intact and readable from the digital media upon which it is
stored. Renderability refers to the translation of the bit stream into a
form that can be viewed by human users, or processed by computers.
Understandability involves providing enough information such that the
rendered content can be interpreted and understood by its intended
users.
OCLC (2002)
In addition to descriptive metadata, a preservation metadata schema will include
components of administrative, structural and technical metadata. Administrative
metadata includes information about the provenance and rights associated with the
digital resource. Structural metadata is used to describe the internal organisation of
the items in a digital resource. Technical metadata includes information about the
formats, software and hardware used by the resource.
Several digital preservation metadata schemas and standards have emerged over
recent years, most notably from the digital library community (National Library of
Australia, 1999; CEDARS, N.D.; California Digital Library, 2001; National Library
of New Zealand, 2002). The devolvement of the Open Archival Information System
(OAIS) Reference Model has spurred development of digital preservation metadata in
this context (Lupovici & Masanès, 2000; Online Computer Libraries Center [OCLC],
2002).
Different factors, primarily the drive for interoperability, are behind developments in
e-learning metadata specifications. It is possible, however, to view the problem of
digital preservation as an interoperability problem; that is, how can we ensure that the
digital resources of today can interoperate with the digital resources of tomorrow. The
emphasis in e-learning standards on interoperability is therefore a good basis for
ensuring the long-term survival of e-learning material.
Interoperability is also an issue for the digital library community where the metadata
interoperability standard METS (Metadata Encoding and Transmission Standard -
METS, 2003a) has received international attention (METS, 2003b). METS is
essentially a container format into which descriptive and other metadata can be
loaded. It also includes a native capability for structural metadata. The Library of
Congress, and other organisations, are investigating the use of METS in relation to the
OAIS reference model, highlighting the close links between information needed for
interoperability and that need for preservation (METS, 2003b).
18

In e-learning, the IMS Content Packaging (IMS CP) specification plays a similar role
(Yee & Beaubien, 2003). IMS content packaging is a specification for sending e-
learning material between VLEs, or other learning systems to another in ways that
maximise the opportunities for combining and recombining them in new ways and yet
retaining the essential aspects of their structure that are necessary for their
functioning. IMS CP metadata separates data about the internal file structure from the
various views/applications/aggregations of it. Through the concepts of packages and
manifests and the following of certain rules, chunks of data can be recombined into
new e-learning resources without, say, hypertext links being compromised. New
resources, such as an entire course or individual lessons, can be put together so that
they are completely self-contained and can stand alone. This is the special
contribution of IMS CP.
Yee and Beaubien (2003) have already proposed A Preliminary Crosswalk from
METS to IMS Content Packaging. Linking digital library and e-learning
interoperability standards (and specifications) may prove a good method of bringing
the e-learning community closer to the work on digital preservation being conducted
within the library and archiving communities.
5.3.3 Rights Management Metadata
When creating new resources, it may be necessary to make use of third party material.
If the material is intended to be kept and possibly reused in different ways then this
will have an effect on the type of licence which would be necessary. In particular
having to renew licences when they have run out can be expensive so the
creators/procurers should consider how the resource might be used with what sort of
benefits they would obtain before making decisions on licensing the material.
The basic legal questions which need to be considered for educational materials are
well summarised by Santos, O & Ramos, F, 2004:
• What sort of usage permissions are possible such as reading, printing, playing
and executing?
• What sorts of re-utilisation permissions are possible such as copy, modify,
redistribute?
• What sort of trading permissions are possible such as sell, lend and lease?
• What sort of usage restrictions need to be considered such as user, device,
time and place?
If appropriate, consideration should also be given to the preservation issues
considered in Preservation Management of Digital Materials (Jones & Beagrie,
2002), such as permission for:-
• Preserving content
• Preserving the associated software
• Copying for the purposes of preservation
• Future migration of content to new formats
• Emulation for purposes of preservation
• Dealing with copyright protection mechanisms
19

In the future, it seems likely that machine readable digital rights expression language
(DREL) may be developed to describe the associated rights for learning resources in
such a way that it can be stored with the resource and provide appropriate details for
digital rights management. Friesen et al (2002) have researched the requirements for
such a DREL. They have concluded that currently available languages may not be
appropriate for research and education and that more work is necessary.
The development of a digital rights management methodology for e-learning materials
will help to facilitate reuse. JISC are aware of this issue and have recently funded a 6
six month project into Digital Rights Management.
The OpenCourseWare project (MIT, N.D.) uses the Creative Commons arrangement,
which includes several different styles of licences that can be used to allow effective
sharing of electronic resources while offering protection for both parties. It is
currently designed for US law though the JISC is working on the Share-A-Like
Scheme that will be similar to Creative Commons but adapted for English Law. This
type of licence is expected to be particularly useful for nationally funded projects.
Recommendation 4
A model licence for sharing e-learning material should be developed.
• Particular attention should be paid to protecting the moral rights of the
original creator.
• Licence terms should ensure that standard digital preservation strategies, such
as file format migration, are not precluded.
• Data protection and copyright issues over user feedback (potentially a
valuable source of quality assessment information) need to be addressed.
Currently, the metadata standards do not contain actual licence details though it would
be possible to use an external link to a page about the licence from the metadata.
Alternatively, details could be stored in a separate file with the resource in a content
package.
A high degree of technical expertise is required to package materials using IMS
content packaging specifications or to apply IMS metadata. The further development
of tools, such as RELOAD will help to ease the creation of standards compliant
materials. This will, in turn, simplify the long-term retention and reuse of these
materials.
Recommendation 5
Guidelines for creating reusable e-learning materials have been developed by the
NLN Paving the Way document (NLN, n.d.), Ferl and X4L Healthier Nation (X4L,
n.d.). Work should be undertaken to determine if these and other guidelines can be
amalgamated into a single best practice guide for UK HE and FE.
Recommendation 6
JISC should fund a survey study of e-learning materials in the UK to determine which
are of sufficient value to invest in making them standards-compliant. The study
20

should focus on the costs and benefits of updating materials as compared with
maintaining them in their present form.
Recommendation 7
Responsibility for the creation, delivery and long-term management of e-learning
materials is often fragmented and unclear. Institutions should develop e-learning
material management policies to clarify roles and responsibilities.
21

6 Encouraging Reuse of E-learning Materials
6.1 Adopting E-learning in the Institution
Institutions often try to encourage academic staff to produce or use e-learning
materials. Many colleges and universities require teachers to digitise their course
material and make it available through the institutional VLE (Virtual Learning
Environment), for example. However, e-learning initiatives can easily run into
difficulties. In Theory and Practice of the Virtual University, Cornford and Pollock
(2004) observe that e-learning initiatives can be “confounded by difficulties in co-
ordinating a wide range of actors across a large organisation made up of diverse and
disparate entities (i.e., departments and service units). It is, it seems, the very
institution of the university which is at the heart of the problem”. Making changes and
innovating in such an environment can, as Cornford and Pollock argue, be difficult
and prone to unexpected results as the many subtle relationships within the institution
are altered.
Rogers (1969) argued that innovations are adopted more quickly when they possess
the characteristics of simplicity, compatibility with existing methods and techniques,
and relative advantage in comparison with these established methods and techniques.
The idea of a relative advantage was seen clearly in the three long term FE and HE
projects examined in this study:
• EuroMET produced interactive content designed to address aspects of
meteorology which people in the field felt were important or difficult to teach
using standard methods.
• The SCRAN database allows or users to search for and access images from a
large collection (over 2 million) and it would be difficult to get access to such
a large collection through textbooks.
• COLEG works on a consortia basis where each college involved contributes a
course unit. In exchange for this the college receive access to peer reviewed
course units created by other colleges so they may receive 15 units in
exchange for submitting one.
Like any other design of electronic learning materials it is important to consider what
clear advantage a particular learning object or collection of learning objects will have
over traditional methods:
• Digital resources can be easier to transfer, store and access than paper based
materials.
• If designed to be reusable, materials may be easier to use by others if original
creators leave.
• It may be justifiable to spend more money to produce higher quality materials
if they will last longer and be used by more people.
• It may be possible to use digital learning materials to generate income or to
share with other institutions.
22

This idea of a clear advantage is important as the use of learning objects will not
always be the most effective way of providing resources for teachers. Both EuroMET
and NLN focussed on areas which end user groups suggested would be appropriate
for computer based learning. Such analysis should be done of user groups needs
before deciding on whether to use a learning object approach.
Simplicity is important in the actual resource and in the way it is accessed. This
means that during the creation phase of electronic materials consideration should be
given to aspects of usability such as those expressed by (Nielson, N.D.) and (ANSI
Standards Committee on Dental Informatics, 2001). In addition the user interface for
accessing and retrieving the materials should be straightforward and quick to use.
The idea of simplicity also comes out in Boyle’s definition of a learning object in
which it is stated that each learning object should have one learning objective (Boyle,
2002). While all the e-learning community does not accept this idea, by having a clear
objective, use will be more straightforward.
Compatibility with existing methods is important as more familiarity will make use
easier. When producing teaching and learning materials, lecturers will tend to use a
lot of materials from a variety of different sources which may be put together in
different ways or modified.
This type of activity is expressed in the concepts of aggregation and disaggregation in
e-learning objects. The idea being that learning objects can be treated as blocks which
could be put together to produce new learning materials. The Open University
CURVE project (CURVE, N.D.) has created learning objects so that they could be put
together to form new courses. This involved using some “mortar” or “narrative
objects” to join the learning objects together.
If a resource does not have a clear advantage over other non-digital materials, there is
no reason why people should use it. For example a simple simulation of a basic
electrical concept such as Ohms Law would have a very large potential audience,
since the topic occurs in many different subjects taught at FE and HE level. However,
there are many such simulations around and in practice, it may not be necessary to use
a simulation to demonstrate this concept.
On the other hand, a larger more complex simulation of acceptance gaps at
roundabouts could be of high value to a relatively small number of people involved in
teaching transport engineering since it may not be easy to demonstrate the concepts
involved without using a simulation.
6.2 Incentives and Requirements for Sharing
Sharing and reusing e-learning materials may lead to an improved quality of teaching,
the sharing of good practice, greater consistency and an enhanced sense of community
(Duncan, 2003). In a questionnaire conducted as part of the JORUM scoping study,
79 percent of respondents indicated they would be interested in sharing their learning
objects through the JORUM repository, and 91 percent indicated they would be
interested in using learning objects deposited in the JORUM repository (JORUM,
23

2004c). There is however, a mismatch between this enthusiastic endorsement of the
JORUM learning repository concept and other, more doubtful sentiments, expressed
by respondents to both this study and the JORUM scoping study:
• “If it’s worth something, why make it free?”
• “Institutions will keep the good stuff for themselves and put the dross online”
• “There are many people interested in sharing, but others [are] interested in
profit”
• “Having learning objects for sale could prove fatal to the sharing ideology”
• “Do we want to follow [the] overpriced text book model?”
Many respondents felt that incentives were lacking to encourage individuals to make
e-learning materials freely available to others. As one person put it, “what is the
incentive for the individual – more work and no gain”.
On a national level many people interviewed have expressed concern that good
materials are not getting as many end users as they should be. In particular, many of
those materials have not been designed for reuse using a learning object approach and
by doing so reuse may be improved.
Both individual and institutional incentives will be necessary to encourage the sharing
and reuse of e-learning materials. Many suggestions about how to create these
incentives were made during both this study and the JORUM scoping study. Given
the difficulty of accurately valuing reusable learning materials (OLCL, 2003, p.8), it
is not surprising that a central feature of most of the suggestions made is the
establishment of a mechanism that will enable e-learning material users, or their
proxies, to judge the value of e-leaning materials and feed their assessment back to e-
learning creators. Possibilities include: the creation of a ‘prestige’ economy, direct
monetary reward, credits towards learning and teaching Curriculum Vitae for
individuals, or a system similar to the Research Assessment Exercise. There was no
consensus among respondents to this study about which approach would be most
likely to succeed, but it is clear that any model for sharing and reusing e-learning
materials must, at the least, address a number of underlying considerations, including
Intellectual Property Rights (IPR), the creation and maintenance of resource discovery
metadata, and quality assurance.
6.2.1 Intellectual Property Rights
IPR is a potential issue in two ways. If sharing and reuse is based on some kind of
money based market, then e-learning materials will need to be accompanied by clear
IPR information that can be used to facilitate rights clearance. Even if e-learning
materials are made available at no cost, perhaps through a licensing framework such
as the Creative Commons (Creative Commons, N.D.), which is used by the MIT
OpenCourseWare initiative (MIT, N.D.) for example, the ability to track rights, and
assure authors due recognition for their work will still be important. A particular
complication for e-learning objects may emerge when they are modified – raising the
question of how to apportion authorship.
24

6.2.2 Discovery
Considering the problem of sharing and reuse from another angle, the problem of
locating appropriate e-learning materials can be emphasised. Obviously, in the short-
term if users cannot locate e-learning materials and determine their relevance quickly,
they are unlikely to be used. In the longer-term, this will complicate decisions about
which resources to retain and invest in, as simple measures of value, such as number
of users, will be difficult to apply with confidence.
Developments such as the Learning and Teaching Portal (LTPS, N.D.) should help
resolve the fragmented presentation of learning and teaching resources at the UK
national level, but high quality metadata records will be a vital requirement.
Methodologies for metadata creation were discussed in the JORUM scoping study
and this study:
• “Content authors needed for detailed metadata but assumptions made about
audience’s knowledge. Paper resources do have increasingly cross-
disciplinary metadata”
• “Different metadata tagging is necessary for the same object if using for
different purposes”
• “Metadata methodology is developing and we will have to modify
approaches”
• “Difficulty of transferring knowledge from the creator to the librarian”
The main long-term consideration these comments suggest is the probable need to
periodically update metadata to reflect changes in curricula, qualifications and shifts
in the boundaries between subjects.
6.2.3 Quality Assurance
Quality assurance is arguably the most important prerequisite for the sharing and
reuse of e-learning materials. However it is very difficult to judge quality, and
attempts to describe the purpose and appropriate use of particular e-learning materials
could be complicated by the many possible uses to which they might be put.
One view expressed to the authors of this study was that no attempt should be made to
formally judge the quality of e-learning materials; experience and word of mouth will
suffice to promote the good and discourage the bad. The majority view, however, was
in favour of some type of formal quality assessment. A number of approaches were
suggested. These are outlined in table 2.
Review Method Advantages Disadvantages
Peer Review
Strict procedure looking at
effectiveness of material
• Can be rigorous
• Trusted
• Expensive
• a lot of work may not
be read
Annotation
Annotations provided by
end user
• cheap
• flexible
• can be subjective and
ineffective in
providing information
• may be few end users
supplying comments
25

Star ratings and hit
counters
Star ratings are given by
end users and the number
of times a resource has
been used is counted
• Easy to get
information
• Subjective
• needs to be supported
with annotations
Forum
Community areas,
discussion
• encourages
growth of
communities of
practice
• may be poorly used
Table 2: Peer Review Methods: Adapted from (JORUM, 2004e)
The suggestions range from academic concepts of peer review through to ideas more
commonly associated with electronic marketplaces (such as Amazon and eBay) or
virtual communities. One problem with traditional ideas of peer review is that they are
designed to consider a fairly narrow range of outputs such as books, papers and
conference papers that are typically created according to well understood and quite
rigid conventions. How well these ideas can be transferred to the far more flexible
nature of e-learning materials and objects is open to question.
Methodologies such as (EaseIT, n.d.) and (Squires. D, 1997) have been developed for
evaluating computer based learning materials, thought they may not be directly
applicable to learning objects. For example, to work they may require smaller (more
granular) learning objects to be assessed within a specific context, making the
assessment dependent on the context.
In may also be necessary to make judgements based on how reusable they are,
examining how they could be used by people with different levels of knowledge and
how applicable they would be for other subject areas. This sort of evaluation was
examined by Currier and Campbell in the “Evaluating Learning Resources for
Reusability the DNER & Learning Objects Study”
JISC has recognised that there is a need to develop methods for evaluation of e-
learning materials and the e-pedagogy programme has recently commissioned studies
into the evaluation of e-learning materials. A more detailed discussion of approaches
to quality assessment in current use for e-learning materials is provided in the
JORUM Scoping and Technical Appraisal Study: Volume V: Metadata (JORUM,
2003e).
6.3 Promotion of Resources
Many people interviewed have commented on the fact that good e-learning resources
do not always reach their intended audience. As one interviewee noted:
“dissemination is not that easy – communication between staff is not always good.
Different dissemination mechanisms are necessary in different institutions.”
From the comments made by respondents to this study, underlying problems appear to
include:
26

• No e-learning equivalent to publishers’ promotion budgets
• Poor communication between staff in institutions
• Lack of clear responsibility for resource awareness within institutions
This suggests that part of the problem is that there is no well-defined infrastructure for
supporting the use of e-learning materials and that there is insufficient funds for
marketing learning materials. The production of much e-learning material within
short-term projects often means that e-learning materials will only be promote for a
brief time before the project ends.
The communication channels in different institutions will function in different ways
and there is often no clear responsibility for e-learning materials promotion. In
particular, librarians, who have traditionally acted as gatekeepers, may now need to
promote resources more actively (Mclean & Lynch, 2003).
Technology can aid resource promotion through federated search systems, such as the
SMETE website (SMETE, 2003) and portals, both of which offer the potential to
promote a single well-known website as the entry point to a comprehensive listing of
e-learning materials. This approach would be reliant on the creation of consistently
high quality metadata in order that portals know that certain resources exist. This in
turn requires services such as registries, which describe the available resources and
makes this information available through machine-to-machine protocols. JISC
currently funds the Information Environment Service Registry project to explore such
a service (http://www.iesr.ac.uk/).
Recommendation 8
Guidance on quality assurance procedures for e-learning material metadata should be
developed.
27

7 Infrastructure for Retaining E-learning Materials
and Objects
7.1 Technical Architectures for E-learning
Learning is an interactive process; it involves interactions between students, between
students and their teacher, and between students and their learning materials. E-
learning systems must support this range of interaction types and also connect with
management systems for student tracking, assessment, payment and other tasks. The
need to support such a network of interactions distinguishes the technical architecture
for e-learning from other digital environments of interest to HE and FE. Many digital
developments in areas such as repositories, libraries and portals can focus on a single
type of interaction. In contrast, the technical architecture that underpins e-learning
must include services and functionality that can support the entire learning process.
In its white paper, What is the Open Knowledge Initiative, the Open Knowledge
Initiative (OKI), discusses the need to connect three general categories of software
relevant to learning:
• Learning applications ranging from individual quizzing, authoring, and
collaboration tools to suites of such tools that include course management and
learning management capabilities,
• Central administrative systems such as student information, human resource,
and directory management, and
• Academic systems including library information systems and digital
repositories of research and educational materials.
OKI (2002, p.2)
Any technical architecture that supports e-learning needs to incorporate or link to
institutional resources such as digital libraries, record management systems, and
collaboration and communication tools (such as video conferencing) in order to form
what is called a Managed Learning Environment (MLE). The recent Managed
Learning Environment Activity in Further and Higher Education in the UK study
explains the term MLE as referring to “the whole range of information systems and
processes of an institution (including its VLE if it has one) that contribute directly or
indirectly to learning and the management of that learning” (The Social Informatics
Research Unit, Education for Change Ltd & The Research Partnership, 2003, p.14).
Combined, and properly integrated, the three categories of software identified by OKI
provide the main components of a MLE. MLEs might not yet be regarded as a mature
technology (Stiles, n.d.) but there is considerable development and use of them in UK
Higher and Further Education (The Social Informatics Research Unit, Education for
Change Ltd, The Research Partnership, 2003).
While a full investigation of MLEs is outside the scope of this study1
it is relevant to
consider how many parts of an MLE will ultimately need to retrieve, access or
1
JISC does have substantial involvement in MLE research and development (JISC, 2001a; JISC, 2001b; JISC N.D.
(a); JISC N.D.(b)). In particular, a range of efforts are underway to integrated digital library developments with e-
learning (JISC 2001c; JISC 2002).
28

manage learning objects. Several groups are working on abstract e-learning system
architectures (IMS, 2003; IEEE, 2003; OKI, 2002). Similar functional themes are
found in each of these architectural models:
• e-learning content creation tools
• e-learning content repositories, including discovery and delivery
• tools for planning, doing, assessing learning
• learner information management systems
These four themes recognise the need to create, store, find and deliver learning
objects and also to track and record the by-products of learning, such as the outcomes
of assessment exercises. The technical architecture which has emerged to support
these requirements is underpinned by network and web technologies. E-learning
materials are typically managed by an e-learning specific content management system
running on a server that then communicates with, and delivers content to, Web
browsers running on client computers used by students. At its simplest, this may
involve nothing more than the delivery of static HTML from a Web server to a Web
browser, like any other webpage. Often, however, the client computer will provide
greater functionality than just display of static content. Dynamic content like
simulations, for example, will require program code to run on the client computer.
7.1.1 Virtual Learning Environments
One software component of an MLE already widely used in UK HE and FE
institutions is Virtual Learning Environments (VLEs). VLEs provide a framework of
technical services needed to support online learning.2
According to JISC,
The principal functions that the complete VLE needs to deliver are:
• Controlled access to curriculum that has been mapped to elements (or
“chunks”) that can be separately assessed and recorded
• Tracking student activity and achievement against these elements using
simple processes for course administration and student tracking that
make it possible for tutors to define and set up a course with
accompanying materials and activities to direct, guide and monitor
learner progress
• Support of on-line learning, including access to learning resources,
assessment and guidance. The learning resources may be self-
developed, or professionally authored and purchased materials that can
be imported and made available for use by learners
• Communication between the learner, the tutor and other learning
support specialists to provide direct support and feedback for learners,
as well as peer-group communications that build a sense of group
identity and community of interest
• Links to other administrative systems, both in-house and externally
http://www.jisc.ac.uk/index.cfm?name=mle_briefings_1 (15/12/2003)3
2
Learning Management System is another, almost synonymous term (Paulsen, 2002, p.6)
29

VLEs have been recognized as a key technology for delivering courses and
programmes (Katz, 2003; Gallagher, 2003), have appeared on the 2003 list of top 10
issues that Chief Information Officers and Information Technology Executives have
to resolve for a campus’ strategic success (Crawford et al., 2002), and their use cited
as an effective practice for redesigning courses for increased quality and cost savings
(Twigg, 2003).
A VLE supports the administration of courses, the delivery of learning materials and
facilitates communications between students, and between student and teacher.
Content creation is not one of the core functions of a VLE, although some do provide
integrated authoring tools. For a comparison of some common VLEs, see CHEST
(CHEST, 2002). VLEs must be able to provide controlled access to e-learning
material stored within them, so the ability to create courses from individual ‘chunks’
of content does fall within the functional requirements of a VLE. Individually, “VLEs
are NOT neutral in their impact on course design” (Stiles, N.D.), and the ability to
extract content from a VLE for use in a different pedagogical environment is therefore
very important.
7.1.2 Interoperability between VLEs
Case studies by CETIS, the CO3 project and the Colchester Institute (Bacon, 2003)
highlight many differences between VLEs that can make sharing e-learning content
between systems difficult. The CETIS “Content CodeBash: Final Report” (CETIS,
2002) describes a number of VLE interoperability issues. These include IMS content
packaging or Metadata version incompatibility issues; Zip file format
incompatibilities; an inability to handle sub-manifests of content packages; an
inability to handle IMS Metadata record in a manifest resource section; an inability to
handle IMS Metadata records outside of the manifest; and rejection of absolute URLs.
Individually, these issues are easy to resolve, requiring manual editing to correct
problems in existing documents, and the implementation of specific work practices to
avoid them in the future.4
Too many of these problems, however, will create a burden
for anyone trying to transfer e-learning materials between different VLEs. Colchester
Institute’s report is typical of many in concluding that:
Specifications are insufficiently developed/defined to enable one
vendor to develop their system to work with another vendor’s products
purely on a basis of the current IMS documentation.
Bacon (2003)
The IMS Question and Test Interoperability (QTI) specification (IMS, 2004) provides
a good illustration of the types of problems that occur. IMS QTI provides a standard
format for specifying, delivering and recording assessment information so that it can
be shared across VLEs and other learning systems. Unfortunately, support for this
specification is not always consistent (CETIS, 2002; Sclater et-al, 2002). The WebCT
VLE, for example, can purportedly export a course with questions stored in
accordance with the QTI specification, but although the resulting data is valid XML,
it does not comply with the QTI specification (Sclater et-al, 2002). Exported content
3
Like most new terminology, VLEs have been defined in a variety of both narrow and broad ways. See Dillenbourg,
2000; Whatis.com, 2003 for two of the many alternatives
4
Ensure the filenames are W3C RFC 2396 compliant, the manifest is at root, and avoid the use of problematic or
illegal characters which can produce invalid URIs (CETIS, 2002).
30

from the Blackboard VLE also demonstrates incompatibility with other VLEs by
storing questions in a proprietary format within an IMS content. Sclater et al (2002)
state that neither WebCT nor Blackboard can import QTI conformant content.
There are noticeable differences between applications in how they import, render and
export question and test content. Although the majority of packages were able to
import simplistic HTML question and answers, only limited success was achieved
when displaying bold text and inline graphics. Perception, Canvas Arena and the
CETIS online tool had only limited success in rendering a bold section of text, while
Perception and Canvas Arena did not fully render inline graphics (Sclater et-al, 2002).
Until standards have stabilized and conformance programs are in place to prevent
vendors from interpreting specifications differently, using third party ‘middleware’
products like Respondus, a quiz and test editor which supports both WebCT and
Blackboard is an effective strategy for circumventing interoperability problems
(Lamberson & Lamb, 2003).
Events such as the CETIS Codebash (CETIS, 2002) represent positive approaches
which suggest that these problems will be reduced with time.
7.1.3 VLEs and Web Browsers
Many VLEs rely on standard web browsers to display content. The benefit of this
approach is that it ensures that the vast majority of VLE users will be able to view e-
learning materials without having to install additional software (a particularly
important issue in universities and colleges where users are routinely prevented from
installing software for security and legal reasons). There are, also, disadvantages.
Web browsers still do not all interpret and implement the relevant standards (HTML,
XHTML, CSS etc.) in the same way, creating differences in the way content may be
displayed (Budd, 2003). Web browsers rely on ‘plugins’ and other software to support
additional functionality. PDF files, for example, can only be displayed within a Web
browser if Adobe Acrobat Reader or similar software is installed. Similarly, popular
video formats (DIVX, Quicktime, and others) cannot be played without additional
decoder software. Thus e-learning content intended for display in a Web browser
must be created with some regard for the formats that are likely to be supported by
users’ Web browsers. For example, a relatively uncommon file format, such as MNG
(Multiple-image Network Graphics – an animation format) can be displayed in the
Netscape and Mozilla Web browsers (with the use of Libmng reference library) but
not in the most widely used browser, Internet Explorer (Libmng, N.D.).
7.2 Digital Repositories, Archives and Libraries
Digital repositories or archives (the two terms are used fairly interchangeably) are a
central feature of discussions about infrastructure being conducted in a number of
areas, including data archiving, digital libraries, e-prints, institutional repositories and
e-learning.
There is general agreement that digital repositories or archives are store places for
digital resources. Users may search them, and retrieve information from them. The
IMS Abstract Framework: Whitepaper refers to content repositories and catalogues
(IMS, 2003, p.14), and similarly the IEEE Learning Technology Systems Architecture
31

refers to “stores” (IEEE, 2003, p.22). In this vein, the IMS Digital Repositories
Specification (IMS DRI) defines a digital repository as:
Any collection of resources that is accessible via a network without
prior knowledge of the structure of the collection. Repositories may
hold actual assets or the metadata that describes assets. The assets and
their metadata do not need to be held in the same repository
(IMS, 2003a)
This view of a repository or archive as a networked system of hardware, software and
metadata designed to store and deliver content is widely held, but it is not the only
view, as the Open Access Initiative (OAI), which advocates improvements in the
availability of scholarly communications, acknowledges:
Members of the archiving profession have justifiably noted the strict
definition of an “archive” within their domain; with connotations of
preservation of long-term value, statutory authorization and
institutional policy. The OAI uses the term “archive” in a broader
sense: as a repository for stored information.
Open Archives Initiative (2002)
As this quote suggests, the archiving and library communities promote a different, and
more demanding, view of digital repositories. A joint OCLC and RLG working group
concluded that:
A trusted digital repository is one whose mission is to provide reliable,
long-term access to managed digital resources to its designated
community, now and in the future.
RLG (2002, p.5)
Internationally, the most influential work to date on how this type of archival digital
repository should operate has been the OAIS (Open Archival Information System)
reference model (CCSDS, 2002). The OAIS model, which is now an ISO standard,
was developed to improve the preservation of data collected during space missions,
but it has found its way into most discussions about digital preservation. An OAIS is
described as:
An archive, consisting of an organization of people and systems, that
has accepted the responsibility to preserve information and make it
available for a Designated Community.
CCSDS (2002, p.1-1)
This model describes in (at nearly 150 pages) considerable detail the six main
activities a digital archive must undertake – ingest, archival storage, access, data
management, administration and preservation planning:
Ingest
32

Ingest includes the physical transfer of files and the legal transfer of rights through
the signing of licences or other agreements that establish the OAIS repository’s
right to maintain the ingested material. During ingest, descriptive information
(resource discovery metadata) should be created to describe the material, and the
submitted files are checked to ensure that they are consistent with the OAIS
repository’s data formatting and documentation standards. This may include tasks
such as file format conversions or other changes to the technical representation
and organisation of the submitted material.
Archival Storage
This functional entity is concerned with the storage of the submitted digital
material, including tasks such as backup, mirroring, security and disaster recovery.
Access
All the services and functions needed for users to find and access the contents of
the repository.
Data Management
Data management involves the collection, management and retrieval of resource
discovery, administrative, structural and technical metadata about the OAIS
repository’s content.
Administration
The administration functional entity involves the entire range of administrative
activities that an archival organisation should undertake. Notable tasks include
managing, monitoring and developing the repository’s software systems,
negotiating submission agreements with producers (authors), and the
establishment of policies and standards for the repository.
Preservation Planning
This functional includes four sub-entities associated with identifying risks that
could affect continued access to the OAIS repository’s holdings, and developing
plans to address them:
Monitor Designated Community – the designated community is an OAIS term that
refers to the community of stakeholders who have an interest in the content of the
repository. An OAIS repository needs to monitor its designated community’s
adoption of new technology, and other trends that may affect preservation of the
community’s digital output.
Monitor Technology – The monitor technology function ensures that the OAIS
repository is constantly aware of technological changes that may render its current
holdings obsolete or difficult to access.
Develop Preservation Strategies and Standards – The development of strategies
and standards for preservation that are informed by the current and future
requirements of the contributors to, and users of the OAIS repository.
Develop Packaging Designs and Migration Plans – This function accepts
standards for file formats, metadata and documentation (generated as part of the
33

administration functional entity) and creates tools or defines techniques that apply
these standards to submissions.
Recognising the distinction between archival digital repositories and what we might
call, for convenience, transitive digital repositories, helps to clarify the different ways
in which the term is used. Transitive digital repositories emphasize the online
discovery and delivery of digital resources, and are therefore often closely associated
with issues of interoperability. Archival digital repositories emphasize the long-term
preservation of digital resources, and this use of the term is often associated with
broader organisational requirements.
7.3 A Model for E-learning Repository Services
Within educational institutions digital repositories do not have a single home.
Specialist centres, departments, or technical support services may administer them.
They may also be run by the institutional library service. When this is the case, the
term digital library is likely to be used instead of digital repository or digital archive.
Like many other digital repositories, digital libraries do not necessarily include
preservation in their remit. Pinfield describes the digital library as “a collection of
networked digital information resources and associated technical and managerial
infrastructure” (Pinfield, 2001), a definition which shares much with other definitions
of transitive digital repositories.
Traditionally, libraries have acted as institutional store places for research and
learning resources. Digital libraries and institutional repositories expand this role into
digital materials, while maintaining ties to the existing expertise and managerial
infrastructure of an institutional library service. Unfortunately, despite what might
seem to be a central position within the institution, developments in the digital library
are not always well coordinated with e-learning developments, leading to the potential
for duplication of effort and fragmented service provision to staff and students:
Bibliographic references that may be included with online course
materials are not automatically hot-linked to the library’s online
catalogue; in some instances, they cannot be linked. The digitally
reformatted materials produced by the library, and the many more such
materials produced at other libraries, museums, and archives, and to
which most libraries now link through various means, are placed
outside the pedagogical purview as it is defined online by instructional
technologies. The digital library as a learning resource is therefore put
at risk. Meanwhile, faculty and students work in isolation from a
wealth of well-organized, high-quality information that is directly
relevant to their learning and for which they have already paid.
Greenstein & Thorin (2002)
Librarians are, however, heavily involved in the development of the emerging concept
of institutional repositories. Institutional repositories are intended to capture, preserve
and disseminate a university’s collective intellectual capital (Crow, 2002; Dspace,
2002), although at the present time, the focus is on capturing research literature
(SHERPA, N.D.). Mclean & Lynch (2003) describe institutional repositories as:
34

archival, stewardship and dissemination systems for content that have a
fairly heavy policy component in terms of who can deposit, what
metadata is required for deposit, acceptable formats and the
implications of format choices for institutional preservation guarantees.
Mclean & Lynch (2003)
In contrast, the IMS DRI specification “is a much lighter-weight concept from a
policy perspective, but specified more tightly from a technical point of view” (Mclean
& Lynch, 2003). Long (2004) notes that while there are a few institutional digital
repository projects there is a “a rapid proliferation of lightweight digital repositories”
designed to hold “ephemeral learning assets that characterize the majority of what
faculty and students use in online learning environments”.
These lightweight or transitive repositories are unlikely to be suitable for long-term
retention. The JORUM scoping study concluded “It is not possible that the same
repository system would answer both transient and archival needs, and further work
would need to be done to create another system to host materials for long-term
preservation” (JORUM, 2004). The JORUM repository should not yet be seen as an
archival repository for e-learning material. The NLN repository plans are based on an
anticipated lifespan of only three to five years for their e-learning materials – after
which it is anticipated that new materials will be created. With an expected lifespan of
only three to five years, few, if any, problems associated with long-term retention will
occur. Thus, like the JORUM repository, the planned NLN repository is unlikely to
have a long-term archival role.
A core set of non-functional requirements for an archival digital repository can be
distilled from the RLG and OCLC working group report, Trusted Digital
Repositories: Attributes and Responsibilities (RLG, 2002), and the OAIS reference
model (CCSDS, 2002. pp. 3-1 - 3-5):
• Obtain sufficient control of the deposit material to permit preservation actions
such as storage, duplication and migration
• Demonstrate financial sustainability
• Provide services within a viable organisational setting, including an
appropriate legal status, mission and staffing level
• Perform the functions of the repository according to documented policies and
procedures that are monitored and can be externally assessed
• Perform the functions of the repository according to relevant standards and
best practices
• Accept responsibility for the long-term preservation of material deposited in
the repository
Within the library and archive community there is a growing awareness that in the
digital world responsibility for preserving information will need to be distributed in
new ways (Crow, 2002a; Beebe & Meyers, 1999; Lynch, 2003). The long-term
maintenance of digital material draws upon a range of skills that may be spread across
a number of specialities and departments within an institution, particularly those
involved with library services and information technology. Some specialised
knowledge may not be available at all within the institution, and the institution may
35

need to involve a third party. Whilst much of the theoretical and practical work to date
has been in the areas of e-prints and so-called institutional repositories, it can also be
applied to e-learning repositories.
The way forward envisioned by many is to disaggregate the tasks undertaken by a
digital repository, so that not all repositories need undertake all tasks.
Fundamental to implementing this disaggregated model is the logical
separation of the content and service components…. This separation
allows for distributed … content repositories to be maintained
independently of value-added services fulfilled discretely by multiple
service providers.
Crow (2002)
Digital preservation can be seen as one of these ‘value-added’ services, and could be
provided in a number of ways, as is suggested in the JISC Continuing Access and
Digital Preservation Strategy 2002-5 (Beagrie, 2002, p. A13).
The JORUM scoping study concluded that it is unlikely that a single central
repository would be able to collect or even physically hold all of the available
learning objects in a given area, and thus a decentralized or “distributed” model is a
more likely scenario (JORUM, 2004). The authors of this study, and those of many
other studies (PADI, N.D.) agree with this conclusion, especially for the development
of archival digital repositories. A centralised repository would face numerous
conflicting pressures, and its establishment would require considerable concentrated
investment. Spreading responsibility for repository services will provide a more
flexible and resilient model.
Figure 2 presents a simple view of the major functions and transactions relevant to the
long-term retention of e-learning materials. The model presented in figure 2 does not
presuppose any specific allocation of responsibilities to organisations. It can be
applied at varying scales, from institutional to national, and it could be implemented
in many different ways. For example, an e-learning repository could undertake all the
activities itself, could work collaboratively with a group of other repositories, or could
rely on external agencies to provide some services. Distribution of tasks has added
advantage of allowing e-learning content to benefit more readily from research and
development in digital preservation techniques currently occurring in other fields.5
5
The Web site Preserving Access to Digital Information, provides a comprehensive
view of current and past activity in digital preservation (PADI, N.D.)
36

D a t a S t o r a g e
D a t a S t o r a g e
R e s o u r c e
C r e a t o r /
M o d if ie r
D is c o v e r y
S y s t e m
D e liv e r y
S y s t e m
m a k e r e s o u r c e
a v a ila b le
R e s o u r c e
U s e r
q u e r ie s
d e liv e r s
P r e s e r v a t io n
S e r v ic e s
m a k e r e s o u r c e
a v a ila b le
'u p d a t e s '
r e s o u r c e
h a r v e s t s
'u p d a t e s '
r e s o u r c e
a le r t s
d e liv e r s
Figure 2: Functional Infrastructure for the Long-term Retention of E-Learning Materials
Apart from the usual requirements of back-up and data storage, the long-term
retention of e-learning objects, or other e-learning materials, will require three main
additional services:
• Provision of digital preservation services, such as migration
• Periodic review and updating of metadata, to ensure pedagogical relevance
• Complex version handling
With regard to the first of these points, e-learning is no different from any other
digital environment. The long-term maintenance of digital content, of any kind,
requires active management to overcome the problems of media degradation and
(more important) technological obsolescence.
The second and third points are more pertinent of e-learning alone. They rest on an
assumption that changes to curricula, and more general changes in approaches to
learning and teaching, may lead to a requirement to update and modify old e-learning
materials and the metadata that describes them so that they better fit contemporary
practices.
7.3.1 Versioning
Previously the importance of holding additional data such as retrieval metadata, IPR
details, evaluations alongside the actual resources was considered. One way of
dealing with this may be to hold all the information in a content package, which can
then be transferred between systems. However, this could cause problems as there
will be cases in which metadata, IPR details or evaluations become updated so the
package will become obsolete.
37

These sorts of problems could be addressed by a unique identifier and resolution
service. Work has been done by UKOLN, CETIS, DCMI and the IMS into unique
identifiers. At a meeting organised by CETIS in London on 21st
October 2003, several
questions were raised:
1) Is it necessary to consider the long term preservation of digital resources and
their identifiers?
2) How are identifiers allocated?
3) How are different versions of a resource managed, identified and related to
one another?
4) What is the cost of identifier systems?
5) What should resolution services do?
6) Is there a simple, workable and easily manageable solution?
There is a common recognition that unique identifiers could have a significant role in
the long term retention and reuse of learning objects. In particular, they could help
with versioning and avoiding multiple appearances of the same resource in a federated
search.
This approach was suggested in the focus group as one attendee was interested in
converting an existing project, called CALARKS, into learning objects:
http://homepages.ed.ac.uk/calarks/arks/materials.html
This process would involve:-
1. Getting legal clearance for modification, reuse and distribution of materials
2. Conversion of files into word format so that people can edit them more easily
or make use of parts of the material.
3. Decrease the size of the granules and add metadata
4. Deposit the learning objects in the JORUM repository.
There have been a lot of electronic learning materials from past projects, such as
Calgroup, Stomp and EDEC in the TLTP programme, which have produced large
quantities of interactive content. Much of the content is high quality (e.g. EDEC has
performed well in Easeit Eng Evaluations) but currently may not be being used to its
full potential.
Consideration should be given to updating such materials so that they may be more
effectively reused. This may involve changing file types so that the resources can be
used on more machines, arranging into chunks of learning materials and tagging with
metadata so that the resources can be found more easily.
Before such work could be undertaken the costs and benefits must be analysed.
Potential benefits may be easier dissemination of materials, easier reuse of materials
and easier tracking of use. The costs involved may be high – Technical skills would
be required to convert the materials, rights clearances would have to be obtained for
parts of the materials and metadata tagging would require librarian skills.
The location of storage for the material would also have to be determined along with
the management of digital rights since many past materials (e.g. EDEC) are currently
used to generate revenue. In addition intended end use would have to be considered
38

and it may be necessary to design all the resources so that they can be used in the
most common VLEs such as Blackboard, WebCT, Technical and Learnwise.
7.3.2 Costs over the Lifecycle
There are clearly high costs associated with designing, managing and reusing e-
learning materials. For example high quality interactive materials such as those
produced by the NLN were quoted during an interview as costing between £13,500
and £22,000 for one hour of material.
The typical length of time over which a learning resource is useful depends very much
on the subject, level and end user group. Several resource creators who were
interviewed have commented that five years is a useful estimate for the lifetime of a
material. During and after the objects lifetime, the constituent assets may be used to
make other resources and the pedagogical ideas could also be useful for creating new
objects.
Many resources will have a longer useful lifetime (e.g. University 1st
year
mathematics might not change much over a 10 year period). On the other hand more
specialised courses such as an MSc in transport engineering may depend heavily on
government policy which may change significantly over a much shorter period.
Frequency of resource modification will also vary as specialised interactive materials
such as NLN will be expensive to modify (NLN modify once in lifecycle) whereas
simpler Word files which lecturers use may be modified frequently.
During the creation phase costs may be associated with the high levels of skill
required to produce materials which can be reused. For example a high level of
technical and pedagogical expertise may be required to produce simulations. There
are also costs associated with obtaining IPR clearance. For example, (McCracken, R
and Gilbart, M., 1995) estimate that for multimedia projects about 10% of the budget
may need to be allocated for obtaining rights.
Recommendation 9
Currently it is often difficult to quantify the cost of collection management because
there are a lot of people and processes involved. Institutions could track costs of
resources management in a more coordinated manner. JISC should seek to fund a
comparative study investigating current collections projects such as SCRAN, HLSI
and the RDN hubs in order to obtain information about costs associated with
managing e-learning materials.
Cost of resource management over its lifecycle must also be considered. This would
include costs of metadata and alterations. Checking IPR, usage and promotion of the
resource would be additional costs.
Consideration must be given to the cost of long term storage for certain scenarios. A
librarian noted that “For long term storage, there is a definite cost. It is difficult to see
where this would be applied. One method would be the individual resource creator
pays. Another way would be for funding councils to give more money to make sure it
happens”
39

Another interviewee commented on the importance of location and funding for long
term storage
• “JISC could not hold materials for long period. It would have to be done
institutionally or by library”
• “Institutional archiving is possible. We could also have a centralised system
which people use as a service”
It is also necessary to do some more cost benefit analysis for the whole process and
this could be done by getting estimates on the number of people using resources and
specific studies looking at how effective the resource use has been. For example,
Boyle has conducted evaluation studies into the use of objects for computer science
(Boyle, N.D.). Such studies need to be supported as there is currently insufficient
evidence to make judgements on strategies for resource creation and management.
40

8 Digital Content in E-learning
Potentially, almost any type of digital content can be used in e-learning – text,
datasets, images, audio clips, video clips, simulations or other types of executable
files.
Various learning styles are best engaged by using a variety of media to
achieve learning outcomes. Selection of media may also depend on
nature of content, learning goals, access to technology, and the local
learning environment.
American Distance Learning Consortium (2003)
Digital content used in e-learning is therefore likely to be stored using a wide range of
file formats; from relatively simple static data formats, such as plain text and GIF,
through increasingly complex data formats such as Adobe PDF, Microsoft Excel and
Apple Quicktime, and on to programming languages such as Java. Maintaining access
to the content and functionality encoded in such a variety of file formats poses a
considerable challenge.
While specifications exist for defining and describing the interactions between
learning objects, the actual digital content held within a learning object is outside their
scope. E-learning systems are designed to work with collections of suitably wrapped
digital content; e-learning systems do not directly manage the internal organisation of
the learning objects. To give one example, “if you package AutoCAD drawings in an
IMS Package and send it to someone else, they will still need to have a program
capable of viewing AutoCAD files” (Wilson & Currier, 2002).
In short, there is an extremely important distinction to be made between the internal
operation of a learning object and its external interoperability. E-learning
specifications and standards will facilitate the latter, but not the former.
Learning objects represent a bundle of information that is ultimately stored as binary
data. If learning objects are to remain useful in the long-term, a critical question is
whether the data within a learning object can continue to be correctly rendered
(displayed, played-back, interacted with) into meaningful content such as text,
images, video clips and exercises. Rendering static content like text and images
involves converting the binary data into a result that can be viewed by a user.
Dynamic content like audio and video clips must be played back, a more complex
form of rendering, while interactive content, like tests and simulations, involves yet
more complex forms of rendering.
Because of the pace at which information technology develops, it is quite possible that
the content held in learning objects may become technologically obsolete while it is
still pedagogically valuable. Dynamic content, interactive content, or content that is
encoded using complex algorithms (such as audio and video) are all particularly
vulnerable, because they rely on sophisticated and complex software to be rendered
correctly. Once the original formats and software used to create the content pass out
41

of use, it will be very difficult to generate an entirely accurate reproduction of the
original rendering of the content (Digital Bewaring Testbed, 2001, p.5).
There is no single definitive approach to ensuring the long-term survival of digital
content. The functionality required to render binary data into meaningful content
changes depending on the period of time being considered. In the short-term, file
formats need to be easily interoperable and supported by affordable software that will
run on a wide range of platforms. In the medium term, backward compatibility from
newer software and import/export facilities to other software packages becomes
important. In the long-term, file formats need to be well defined, and preferably
conformant to accepted standards, so that they can be successfully migrated. Software
needs to be well constructed and documented so that it can be ported to a new
environment or run under an emulator.
Both migration and emulation – the main approaches advocated for the long-term
preservation of digital content – rely on file format specifications being known and
accurate. Without accurate file format specifications, these strategies risk introducing
distortion, loss of quality or data, or even complete failure to render the content
(University of Leeds, 2003, p. 5). Even when format specifications are available, they
are not necessarily unambiguous. Experimenting with Adobe’s popular Portable
Document Format (PDF), format, (Ockerbloom, 2001) found it necessary to refer to
the actual display generated by Adobe’s own Acrobat software in addition to the
published format specification for the format in his attempts to replicate the rendering
of PDF files.
The task of adequately rendering digital content is made easier if there is agreement
between the creators, users, and managers of the content about what constitute its
significant properties (Cedars Project, 2002). The significant properties of digital
content are those aspects of how it is rendered that are regarded as crucial to its
correct use and interpretation. For example, the author of a plain text document
cannot specify the typeface or font of text, so information indicated through, say the
use of italicised text to indicate emphasis, cannot be transmitted to the user of the
document, and so typeface and font should not be treated as significant aspects of how
the file is rendered. In contrast, the author of a Microsoft Word document can store
information about the typeface and fonts used in the Word file and expect users to see
the same typeface. So, in this case, typeface and font could form part of the significant
properties of the document, as they do in this document (in either its Microsoft Word
or Adobe PDF renderings).
The concept of significant properties is widely used in the digital preservation
community and is implicit in many of the ways digital resources are presented. By
ignoring unimportant aspects of the way digital content is rendered, the future task of
recreating that rendering is made simpler. The JISC Continuing Access and Digital
Preservation Strategy states “there is a need for explicit and open policy and
procedures for preservation and definitions of ‘significant properties’ of objects to be
preserved” (Beagrie, 2002, p.12). In the e-learning field, the debate over pedagogical
aspects of e-learning may provide the starting point for identifying the significant
properties of e-learning materials. Although not usually presented in these terms, e-
learning interoperability initiatives may also serve as forums for clarifying the
significant properties of learning objects.
42

The development of specifications and standards for e-learning has reached a point
where it is possible to package digital content and metadata into learning objects that
can be shared between digital repositories, VLEs, and other systems. Although these
specifications and standards are not primarily intended to support the long-term
preservation of digital content, they do encourage interoperability and, in some
specific situations, software independence (for example, the use by SCORM of an
application programming interface to reduce exposure to changes in web browser
software). These are both factors that will aid the long-term maintenance of e-learning
materials.
Recommendation 10
E-learning repositories should establish practical definitions of the significant
properties of different types of e-learning content. These might be expressed through
restrictions on acceptable file formats, guidance on how to use particular formats, and
requirements for particular metadata to accompany content.
Recommendation 11
E-learning repositories should consider limiting the range of file formats they will
accept within learning objects or other types of e-learning materials.
8.1 Long-Term Risks to Digital Content used in E-learning
There are a range of factors that suggest digital content is more likely to survive in the
long-term:
• The content is simple, rather than complex
• The content is stored using file formats based on open or freely published
standards
• The content does not utilise proprietary extensions to otherwise open standards
• Software that can correctly render the content is easily available
• Software that can correctly render the content is available from a number of
vendors
• The content is stored in a popular, widely used, format
• The significant properties of the content can be represented in more than one
file format
All of these factors can and should be considered by those creating or repurposing
digital content for use in e-learning. In the light of these criteria, this section of the
report seeks to provide an illustrative, but not exhaustive overview of the problems
that may affect common types of digital content in the long-term.
8.1.1 Plain Text
Plain text is one of the simplest and most easily preserved types of digital content.
Plain text consists of characters that represent letters, punctuation, line breaks and
other standard typographical elements. Each individual character is represented by a
character code, and a defined set of character codes are known as a character set or
43

repertoire. These codes may then be encoded in variety of ways into the binary values
that are actually stored in a text file (in much the same way that a given number can
be written in decimal format, as a fraction, or using scientific notation). Correctly
rendering plain text involves both knowing which character repertoire has been used,
and how the character codes have been encoded in the file.
In the past, the main problem encountered when rendering plain text was confusion
over exactly which character repertoire should be applied to the text file. Even the
very widely used ASCII (American national Standard Code for Information
Interchange) character repertoire was subject to some national variants (Czyborra,
1998) to allow for inclusion of regional symbols, such as the pound sign (‘£’). ASCII
now forms a subset of many other character repertoires that have been created for
computers running DOS, Windows, Macintosh and other operating systems. The ISO
standard, ISO 8859-1 (Latin 1), which shares many characters and codes with ASCII,
is widely used for defining the character set used in web pages. Internationally,
Cyrillic, and Asian languages, among others, introduce more problems. Correctly
rendering plain text can be a surprisingly complex problem unless the character
repertoire and encoding are known (Korpela, 2004)
An effective solution to these problems now exists in the form of UNICODE.
UNICODE endeavours to provide separate character codes for each distinct character
from all of the world’s languages, avoiding the need to reuse the same code to
represent different characters. UNICODE is now widely supported, and is backwardly
compatible to ASCII. Difficulties may still occur though, as users will not necessarily
have appropriate fonts to display all these characters installed on their computers, and
UNICODE is somewhat more complex than previous character encoding schemes as
it supports multiple methods of encoding (UTF8, UTF16, UTF32).
8.1.2 Formatted Text: Binary Text Formats
In addition to displaying plain text, with the potential pitfalls just discussed, word
processors and other applications typically support sophisticated layout and
presentation options for text. The two most popular file formats for storing this type
of content are Microsoft Word and Adobe Portable Document Format (PDF).
The binary text format most closely associated with the Web is Adobe’s Portable
Document Format (PDF). PDF combines attributes of structured text formats and
traditional image formats, so that a document can be used both in terms of its text and
in terms of its displayed layout on-screen or on a page. The PDF specification is
published by Adobe (Adobe Systems Incorporated, 2003a, File Format
Specifications) and this has led to a good number of third party tools being developed
to work with this format (PDF-Tools.com, 2003, Planet PDF, 2003). The availability
of PDF documentation and the range of third party tools that exist are often cited as
reasons to be confident that the format will continue to be accessible (James et-al,
2003). The PDF standard is both complex and under ongoing development by Adobe,
and this means that much of the support for the format, as well as the tools that are
considered to set the benchmark for PDF use, do come from Adobe (Ockerbloom,
2001).
The use of PDF as a format for long-term preservation remains a matter of some
debate (Ockerbloom, 2001). Adobe is currently developing the means to express PDF
44

documents as XML, which should provide a better option for long-term preservation
of PDF content, and there is an effort underway to develop PDF specifically as an
archival format (LeFurgy, 2003; Adobe, 2003; AIIM, 2003). However, there is
currently no alternative format that can represent all the features of PDF
(Ockerbloom, 2001). On the other hand, the relative ease of defining the significant
properties of what is essentially a paper surrogate format, combined with its
widespread use mean that PDF files are unlikely to become unreadable in the near
future.
Away from the Web, Microsoft Word is the predominant binary text format in use
today. Although widely used, Microsoft Word, and the other Microsoft Office
formats, are not viewed as formats suitable for the long-term preservation (Heslop et-
al, 2002). Word is a proprietary format subject to frequent changes and Microsoft,
unlike Adobe, does not make the file format specification available, although there are
some third party tools that can read Word documents (University of Leeds, 2003, pp.
24-26). Nevertheless, the superficial ease of use and rich functionality offered by
Word, combined with its ubiquitous presence mean that it is very likely that digital
content used for e-learning will be created in this format.
Conversion to Microsoft’s text-based mark-up format, Rich Text Format (RTF) offers
one way of reducing the long-term risk associated with Word. Microsoft published the
specification for RTF, and provides source code for an RTF file reader. The main
security for content held in the Word format is its widespread use, which mitigates
against the possibility of software for rendering Word files disappearing in the near to
medium term future.
8.1.3 Formatted Text: HTML and XHTML
Digital data is ultimately binary, but a distinction is frequently made between formats
that are based on character encoding schemes and those that store the data directly as
binary values. Data held in so-called binary formats is directly encoded as binary
values, the meaning of which is format specific. Character, or text, based formats,
which are typically based on the mark-up languages SGML and XML can be read as
text, but the text is structured according to a format specification, such as an XML
DTD or schema.
HTML and XHTML are, respectively, the SGML and XML based formats used for
defining Web pages. While the content of an HTML or XHTML file can be viewed in
a text editor, a web browser is needed to correctly render the content.
HTML (HyperText Markup Language) is a relatively simple language that was
designed to define the structure of a document (headings, sections, lists and similar
divisions). HTML has ended up as a defacto layout language for Web pages, although
it was not designed with this in mind. XHTML (Extensible Hypertext Markup
Language) is "a reformulation of HTML 4.0 as a specific implementation of the
Extensible Markup Language (XML)" (W3C). XHTML supports all HTML 4 markup
elements and attributes. Unlike HTML, however, XHTML can be extended through
the definition of new elements and attributes, making possible new ways to embed
content and programming in a Web page, but also effectively creating new formats.
45

A mark-up document’s format is defined in a DTD (Document Type Definition) or
schema. The correct rendering of a mark-up document depends on having software
that embodies a full and accurate understanding of the rules of the format defined in
the DTD or schema, and its accompanying documentation. Particular problems may
occur when part of the format information required for mark-up content is obtained
from a reference to an external (to the marked-up document) resource, such as
Javascript, Cascading Style Sheet, DTD or XML schema. In these circumstances,
specific functionality in the interface is likely to be unavailable and the displayed
results may differ significantly from the intended purpose (e.g. text not formatted in
the correct font or style, graphics and links becoming inaccessible, XML elements
becoming unidentifiable, etc.). The use of proprietary extensions is also common and
can cause significant problems when displaying markup content within different
applications.
The recent JISC reports, Feasibility and Requirements Study on Preservation of E-
Prints (James et al, 2003), the Survey and assessment of sources of information on file
formats and software documentation (University of Leeds, 2003) and An
Investigation into Free E-Books (Berglund et al, 2004) all provide useful summaries
of the long-term issues likely to be encountered with a range of text formats and
software.
8.1.4 Bitmap Images
Bitmap image formats are one of the better understood types of digital object.
Because of the large volume of bitmap images created by digitisation projects they
have been the focus of considerable attention and effort to ensure their long-term
survival (Kenney & Reiger, 2000a, ch. 8; Kenney & Rieger 2000b; TASI, N.D.;
NISO, 2002). They are relatively simple, containing static data that has easily
understood and commonly agreed significant properties, chiefly pixel location, size
and colour. The encoding of this information can be complicated by compression
techniques, Internet delivery considerations (progressive downloads), and the addition
of extra features, such as support for animation or layers. Web browsers, central to the
delivery of much e-learning content, support a limited range of image formats (JPEG6
,
GIF and PNG), and this simplifies the situation. However, it should be kept in mind
that formats such as Word and PowerPoint, delivered as files via a web browser, can
include embedded bitmap image objects in any number of formats.
The popularity of the Web has led to legal problems for the two main Web image
formats, GIF and JPEG. Since 1995, Unisys has enforced a 1985 patent it owns that
covers the LZW compression algorithm used in the GIF image format (Sarrazin,
2003). These patents are now expiring (Unisys, 2004). Unisys has reached over 3,000
licensing agreements for the use of LZW (Unisys 2004) and interestingly, GIF
remains a very popular format, more widely used than the technically superior open-
source PNG format, developed with the intention of providing a free-to-use
replacement for GIF (Roelofs, 2004). More recently, Forgent Networks have
challenged the free-to-use status of JPEG image compression, citing their ownership
of U.S. Patent No. 4,698,672 (Glasner, 2002). Again, although this may lead to
6
JPEG is the acronym of the Joint Photographic Experts Group who developed an ISO standard for image
compression, referred to as JPEG. What is commonly referred to as a JPEG file is actually a JFIF (JPEG File
Interchange Format) file
46

addition costs for software developers, it is unlikely to affect the usage of this format,
given that it now so widely used.
The de facto standard format for storing archival images is uncompressed TIFF
(Tagged Image File Format). The TIFF format is well documented in a publicly
available specification (Adobe Systems Incorporated, 1992) and a wide range of
software can read the format; however there are problems when handling specialist
extensions, such as Adobe’s PageMaker 6.0 (Ritter 1997, Rinke N.D.). TIFF
incorporates support for a range of lossless (LZW/CCITT) and lossy compression
(JPEG from version 6 of the format), and can also contain contextual metadata
including information about the author and copyright.
The great advantage of the GIF, JPEG and TIFF formats is that they are well
established and supported by a huge range of software packages. More sophisticated
bitmap image formats exist, but these tend to be tied more closely to particular
software packages, such as Photoshop and Paint Shop Pro. These formats support
layers, masks and other advanced features not necessary generally supported by all
bitmap graphics formats. These packages typically have extensive import and export
options, and specialist graphic format conversion utilities also exist, so it is not
difficult to transfer content from one file format to another, but there may be some
loss of features.
The potentially large size of bitmap images makes compression, especially for
delivery over networks, an important consideration. Master image files, however,
should be stored in uncompressed or losslessly compressed file formats, otherwise
problems will occur in the longer-term as it becomes necessary to transfer content
from one file format to another. Each lossy to lossy file format conversion will lead to
further degradation of the image quality.
8.1.5 Audio and Audio/Video
Audio and audio/video content have obvious possibilities for learning and teaching.
However, the volume of data generated and the need to play it back, rather than
simply display it, make these forms of digital content more complex than those
already discussed. The encoding of both audio and moving image content is often
complicated by the use of sophisticated compression techniques to reduce file sizes,
and the addition of extra features, such as support for licensing information. Thus,
while the basic significant properties of audio (frequency and amplitude of the sound
wave) or moving images (static images displayed one after the other at a set pace) are
not complex, recording, storing and playing them back on a computer can be a
complex affair.
Audio and moving image content is rendered through the use of a potentially fragile
mix of playback software, formats and codecs (coder/decoders).7
Streaming
(delivering content on-the-fly to a client from a server) introduces further
complications. The loss of any one part of this chain of functionality will prevent a
user accessing the content encoded in audio or moving image formats.
7
Codec is an acronym that stands for "compression/decompression." A codec is a software algorithm, or specialized
computer program, that reduces the amount of storage space required for a complicated file by eliminating redundant
data.
47

The large number of options available for encoding audio or moving image content
means that users may not have all the necessary software installed on their computers.
Software and formats for moving image content are supplied by a range of well
known companies. Usage and support for these formats is affected by their
provenance. Software designed for Microsoft Windows is more likely to support AVI
(Audio Video Interleaved) than Quicktime, an Apple format. In turn, software for
Apple’s operating systems is more likely to support Quicktime than AVI. Alternative
formats, for example Real and MPEG-1 (Motion Picture Expert Group), MPEG-2 and
MPEG-4 are well supported partially because of their platform-independent nature.
The MPEG standards have strong support a known industry standard that can be
played on a variety of platforms using a variety of players. Real, in contrast, is a
proprietary format that first gained a foothold because it offered more advanced
functionality than other formats available at the time, namely streaming highly
compressed content. Even where there are published standards, such as the MPEG-1
and MP3 (MPEG 1 Audio Layer 3), formats based on these standards are not
necessarily fully compatible with each other because proprietary components may be
included (Fries & Fries, 2000). AVI, Quicktime and MPEG-1 are three common
audio/video formats supported by VLEs. These video formats are typically well
documented, providing detailed descriptions of how to produce video data, but differ
in the patents that restrict their use, structure, and user configurability. MPEG-18
is
stable and well supported by WebCT and other VLEs, allowing the designer to utilise
video in its native format, or convert it to RealVideo G2 for streaming from an
appropriate server (Napier, 2003).
In addition to the range of formats available, there are a considerable number of
codecs available for audio and moving image data (Fries & Fries, 2000) Reuse of
content may be stymied by its reliance on outdated or obscure codecs, making the
data inaccessible. Potential problems they may be encounter include: video with no
audio, audio with no video or video that has been rotated 180 degrees. A pertinent
example of these problems was found during this study on the Emporia State
University WebCT Video Tutorials website (Emporia State, 2004) which contains
files in the AVI (Audio Video Interleave) format. The authors were unable to
playback these files in Real’s RealPlayer, or Microsoft’s Media Player because of a
missing codec.
The large sizes of uncompressed audio or moving image files means that much
content, like bitmap image content, is encoded using lossy compression techniques.
This is a potential long-term problem, as the quality of the recording will be degraded
if it is migrated through a series of lossy formats to avoid technological obsolescence.
Master bitmap images are typically stored in uncompressed or losslessly compressed
formats, and the same could be done for audio and moving image data, although file
sizes would be very large.
Lossless compression techniques for audio data do exist, such as Monkey’s Audio
Format (APE) and the Free Lossless Audio Codec (FLAC) format, but they can create
very large files (tens of megabytes for a few minutes of sound). Moving image
content generates even larger files, and there are very view lossless formats for
moving image, because they are not very practical in many situations.
8
The Motion Picture Expert Group is an independent body that have created various video encoding standards that
have subsequently been ratified as industry standards (MPEG-1: ISO/IEC 11172, MPEG-2: ISO/IEC 13818, and
MPEG-4: ISO/IEC 14496).
48

Because of the range of formats and codecs, the long-term survival chances of any
given audio or audio/video content is difficult to determine without detailed
knowledge of the format and the codec, as well as other options selected within the
format, that were used to encode the content.
8.1.6 Questions and Tests
Online assessments are of special interest because the formats and software used are
developed especially for e-learning. There are many online assessment systems
available (Clark, N.D.). Typically, they are client-server systems, with the front-end
taking the form of a Web page displayed in a Web browser on the user’s computer.
This provides the form that the student uses to answer the questions. This is usually a
single or multiple pages that consist of text, graphics, selectable options (radio
buttons, tick boxes, clear and submit buttons) and scripting choices (number of
attempts, time extensions). Additional elements, such as Javascript or Java functions
may also be included with the Web page. The server based element of the assessment
is likely to consist of a relational database and various scripts, written in any of a
number of possible scripting or programming languages, that generate the question
page, establish specific criteria (time limit, IP range, etc.) and submit answers into a
specified database table and field.
To support the transfer of question and test content between different assessment
systems, the IMS have developed the IMS Question and Test Interoperability
Specification (IMS, 2004a). This standard has the potential to act as a universal data
format for question and test content, and a range of assessment software already
supports it.
8.1.7 Simulations
Simulations are executable programs, and this section serves also to introduce some
of the general problems that may occur when the content that must be maintained is
embodied as executable code, rather than static data. Simulations present a
considerable long-term challenge because they are custom written computer programs
designed to run in a specific environment. Most simulations used in e-learning are
designed to be delivered via the Web. Consequently, programming environments
suitable for use with Web browsers and servers, such as Java, Flash and JavaScript,
are favoured. Simulations created using these programming environments can be
embedded into Web pages, making them easy to package and share.
Macromedia’s Flash format and player are a good example of the close association
between data and software used to integrate audio, bitmap and vector graphics into a
single interactive package. The benefit of products such as Flash is their ability to
script movement, interactivity and multimedia content according to user choices. The
disadvantage is that they are proprietary and sensitive to the wider software and
hardware environment in which they are used; they will be affected by the learning
environment in which they are played (Choudhury & Hunter, 2003).
A potential issue for these resources is the need for a plug-in to playback the format
and third-party software to edit it (Macromedia's Flash authoring tool or
Dreamweaver, for example). If Macromedia ceased to support the format, this plug-in
49

would require emulation or be rewritten to execute content on a new operating
system. Although migration paths are available (e.g. export graphics to SVG, audio to
WAV, animation to animated GIF or similar), it is difficult to maintain the interaction
between these elements, which is a key significant property of complex interactive
resources like simulations.
Simulations are complex programs, and major changes are unlikely to be easily
carried out. Minor changes (font, logos, limited functionality changes) could be
carried out if future users have access to the original source code and suitable
interpreters or compilers to create an executable program from the source code. For
simple calculator style simulations, editing the code directly is possible – as long as
there is expertise in the relevant scripting language.
At present there are no e-learning specifications or standards designed specifically to
deal with simulations. IMS is developing the Interactive Content and Shareable State
Persistence (SSP) specifications (IMS, 2004b). SSP would blur the boundaries
between static and dynamic content – as it governs the arrangements for providing
server space to the simulation so that it can store data about its state for retention by a
VLE or MLE. This is a future issue, and relevant for deployment rather than storage.
Standards such as SSP for simulations will mean that there will be an evolution of
simulations to be online and collaborative.
In addition to e-learning metadata such as information on the education level of a
simulation, the long-term retention of simulations would be assisted by the creation of
technical metadata that describes the system requirements of a simulation.
8.1.8 Java Based Simulations
Java provides an ideal way to incorporate the capabilities of a full programming
language into Web delivered content. On the web there are some trivial uses of java
around (scrolling banners, slideshows etc.) but the main use of java applets (Java code
that can be run inside a Web browser) are as games and simulations. Java applets are
easily integrated into web pages. The applets themselves may refer to other files (e.g.
images) but authors are usually quite good at storing attendant files sensibly, so
usually everything would be under one directory structure. As such, creating learning
objects out of the Web page and applet is pretty straightforward - no more so than
creating learning object for a normal web site.
Its traditionally difficult to update or re-engineer Java Applets as they use compiled
java code, rather than the source code (which is often not distributed). Reverse
engineering of the compiled code is likely to breach IPR agreements. The JeLSIM
Simulation toolkit addresses this issue by separating Java coding from interface
creation. Therefore, interface changes (and indeed new interfaces to the original
model) can quite easily be carried out by a third party with access to the appropriate
visual tools and original compiled simulation code and interfaces. This is similar to
Macromedia’s Flash environment, where minor changes to the Flash animation can be
carried out by anyone with access to Flash and the original files.
Like Flash, Java a plugin to be loaded by the Web browser. Unlike Flash, the plugin is
quite large at about 10MB. On computers running Microsoft Windows, Java applet
support is not necessarily installed by default, due to legal arguments between
50

Microsoft and Sun. Although not an ideal candidate for a long-term emulation engine,
the Java Virtual Machine is not without merit, and appears to be the best available
environment, as discussed in the Digital Preservation Testbed project white paper
Emulation: Context and Current Status, (Digital Bewaring Testbed, 2001, p.42). The
principal weakness of Java is its continuing development, which leads to a degree of
instability and backward incompatibility. Java applets (Java code designed to be run
inside a Web browser) can be version 1.1 compliant, or version 1.2 and above
compliant.9
Due to the legal difficulties between Microsoft and Sun, the Microsoft
Java Virtual Machine only supports version 1.1 applets. The prevalence of the
Microsoft Java Virtual Machine means that nearly all applets are designed with the
limitations of Java version 1.1 in mind, but there is potential for incompatibilities.
Sun has had a poor reputation for new versions of its plugin not being fully backwards
compatible (Sun Microsystems, 2003; Nehrer, 2003). Consequently, developers may
be forced to modify their applets following the release of some updates to ensure that
they can be viewed with all versions of the plugin.
Problems with cross-platform support also exist, and this is somewhat surprising
given that Java is often sold on its claim to cross-platform support. Until recently,
support for Java on Apple Macintosh computers was poor. There are now official Sun
versions of the Java Virtual Machine for both Apple’s OS X operating system, and the
Linux operating system, and these have improved cross-platform consistency.
8.1.9 Flash Simulations
Macromedia Flash and Director (and the earlier Authorware and Toolbook products)
are the main alternative tools for creating simulations. Compared to Java they benefit
from easier handling of graphics, but the scripting language supported by these
products is not as strong as Java, and complex simulations created using these tools
tend to become inefficient.
All these tools have similar issues of durability to Java.
8.1.10Other Simulations
For simple, non-graphical simulations (things such as online calculators) much can be
achieved using the Perl, Javascript of another scripting language. This type of
simulation can actually become very sophisticated – see the Biz/ed Virtual Economy
simulator for example (Biz/ed, N.D.). Such simulations tend to be functional rather
than educational. Problems here could include the reliance on server side code and
issues of browser compatibility
Finally, a simulation might be written as a stand-alone executable program in any
suitable programming or scripting language. The long-term preservation of such a
simulation would depend on continued direct support for the programming languages,
so that the source code could be recompiled to run on newer hardware as needed, or
the emulation of the original hardware so that the original compiled code would
continue to run on newer computers.
9
Java 1.2 is, confusingly, referred to as the Java 2 Platform.
51

8.2 Conclusions
Individuals, projects and others creating digital content used for e-learning need to be
aware that adherence to e-learning specifications and standards will not in itself
suffice to protect their products from the risk of technological obsolescence. JISC and
other funding bodies already support a considerable number of activities that provide
advice on best practice and standards relevant to a wide range of digital content
creation. These include services such as the AHDS, the Technical Advisory Service
for Images (TASI), and the Open Source Software Watch, and programme support
activities, such as the New Opportunities Fund Digi (NOF-Digi) programme
Technical Advisory Service and the JISC QA Focus project. Unlike organisations
such as CETIS (the JISC funded Centre for Educational Technology Interoperability
Standards), other services may not be able to provide expert guidance on matters
specific to e-learning, but they can provide advice on other aspects of creating and
managing digital content which are relevant to the creation of e-learning materials.
Recommendation 12
A number of existing advisory and training services are relevant to the creation of
high quality e-learning material within institutions. Relevant services include
Netskills, the newly established Digital Curation Centre (DCC), the Arts and
Humanities Data Service (AHDS), the Technical Advisory Service (TASI) for Images
and the Managing Agent and Advisory Service (MAAS). JISC should consider ways
of targeting promotion of these services at e-learning material creators.
Recommendation 13
The development of e-learning materials to appropriate standards and specifications
will greatly ease preservation issues associated with long-term retention, and will
simplify reuse.
Institutions and JISC should work together to develop a training scheme for tutors and
lecturers interested in developing e-learning materials. The online nature of most e-
learning material suggests that a formal relationship with the Netskills service may be
worthwhile.
Recommendation 14
Institutions should seek to apply any digital preservation work being undertaken in a
library or archival setting with e-learning systems and content development that may
also be occurring.
52

9 References
Adobe Systems Incorporated. PDF/A: FAQ. 2003. Accessed on 12 Mar 2004.
< www.adobe.com/government/pdfs/faq_archivingstandard_pdf_a.pdf >
AIIM International. PDF-Archive. 2003. Accessed on 12 Mar 2004.
< http://www.aiim.org/standards.asp?ID=25013 >
American Distance Learning Consortium. ADEC Guiding Principles for Distance
Teaching and Learning. 2003. Accessed on 11 Mar 2004.
< http://www.adec.edu/admin/papers/distance-teaching_principles.html >
Anon. Learning Objects for Introductory Programming 12 Mar 2004. Accessed on 15
Mar 2004.
< http://www.unl.ac.uk/ltri/learningobjects/intro.htm >
ANSI Standards Committee on Dental Informatics, Guidelines for the design of
educational software. 2001. Accessed on 15 Mar 2004.
<http://web.archive.org/web/20030622201317/http://www.temple.edu/dentistry/di/eds
wstd/title.htm >
Armitage, S. & O’Leary, R. A Guide for Learning Technologists. E-Learning Series
no. 4. LTSN Generic Centre. 2003. Accessed on 17 Dec 2003.
<http://www.ltsn.ac.uk/embedded_object.asp?
id=19513&prompt=yes&filename=ELN060 >
Bacon, J. Public Report To Prove the Concepts of Interoperability Within Managed
learning Environments in the Further Education Sector. 2003. Accessed on 15 Mar
2004.
< http://www.jisc.ac.uk/uploaded_documents/rep-colchester.pdf >
Bailey, M. Learning, Technology and Educational Transformation. 2002. Accessed
on 15 Mar 2004.
< http://education.ed.pacificu.edu/bcis/workshop/adoption.html >
Barker, E. & Ryan, B. Case studies in implementing educational metadata standards.
Mar 2003. Accessed on 14 Mar 2004.
<http://metadata.cetis.ac.uk/usage_survey/cs_hlsi.pdf >
Beagrie, N. A Continuing Access and Digital Preservation Strategy for the Joint
Information Systems Committee (JISC) 2002-2005. JISC. 2002. Accessed on 18 Dec
2003.
< http://www.jisc.ac.uk/uploaded_documents/dpstrategy2002b.rtf >
Beebe, L. & Meyers, B. The unsettled state of archiving. The Journal of Electronic
Publishing, 4(4). 1999. Accessed on 15 Mar 2004.
< http://www.press.umich.edu/jep/04-04/beebe.html >
53

Berglund, Y., Morrison, A., Wilson, R. and Wynne, M. An Investigation into Free
eBooks. 2004. Accessed on 13 Mar 2004.
<http://www.ahds.ac.uk/litlangling/ebooks/index.html >
Biz/ed. Virtual Economy. N.D. Accessed on 13 Mar 2004.
< http://www.bized.ac.uk/virtual/economy/index.htm >
Boyle, T. Design principles for authoring dynamic re-usable learning objects
. 2002. Accessed on 15 Mar 2004.
<http://www.ics.ltsn.ac.uk/Learning_Objects/lmu_learningobjects/papers_pres/ASCI
LITE_slides_2002.ppt>
Boyle, T., Learning Objects for Introductory Programming. N.D.
<http://www.ics.ltsn.ac.uk/Learning_Objects/lmu_learningobjects/evaluation.htm>
Budd, A. Why websites look different in different browsers (or why pixel-perfect
design is not possible on the web). 2003. Accessed on 15 Mar 2004.
< http://www.message.uk.com/articles/display.php >
Calarks. ARKS Material. 2000. Accessed on 15 Mar 2004.
< http://homepages.ed.ac.uk/calarks/arks/materials.html >
California Digital Library [CDL]. Digital object standard: metadata, content and
encoding. 2001. Acccessed on May 3, 2003.
<http://www.cdlib.org/about/publications/CDLObjectStd-2001.pdf>
Campbell, L. Talk given at “The New Economy: Learning Objects” 2003. Accessed
on 15 Mar 2004.
< http://www.intrallect.com/news/seminar_powerpoint/lorna_campbell.ppt >
Campbell, L. Reusing Online Resources, Edited by Allison Littlejohn, Kogan and
Page, 2003.
Casey, J. Draft Document: Intellectual Property Rights and Networked E-Learning,
2003.
CCSDS, Consultative Committee for Space Data Systems [CCSDS] Reference model
for an open archival system, CCSDS 650.0-B-1 Blue Book. 2002. Accessed on 15 Mar
2004.
< http://www.ccsds.org/documents/650x0b1.pdf >
CEDARS (N.D.). Metadata for digital preservation: the Cedars Project outline
specification. Accessed on May 3, 2003.
<http://www.leeds.ac.uk/cedars/colman/metadata/metadataspec.html>
Cedars Project. Cedars Guide to Digital Collection Management. 2002. Accessed on
18 Dec 2003.
< http://www.leeds.ac.uk/cedars/guideto/collmanagement/guidetocolman.pdf >
CETIS. CETIS Content Code Bash: Final Report. 2002. Accessed on 12 Mar 2004.
54

< http://www.imsglobal.org/developers/ims/imsforum/attachments/codebashsum.pdf
>
CHEST. VLE Comparison Grid. 2002. Accessed on 5 Jan 2004.
< http://www.chest.ac.uk/datasets/vle/ >
Choudhury, S. & Hunter, J. PANIC: Preservation and Archival of New Media and
Interactive Collections. 2003. Accessed on 18 Dec 2003.
<http://www.ecdl2003.org/presentations/papers/session11b/hunterchoudhury/ecdl200
3_publish_version.ppt >
Clark, D. Enhancing the IMS Q&TI Specification by adding support for
dynamically generated parameterised quizzes. N.D. Accessed on 13 Mar 2004.
<http://infocom.cqu.edu.au/Staff/Damien_Clark/Research/thesis.html>
COLEG COLEG website N.D. Accessed on 26 Mar 2004.
<http://www.coleg.org.uk/ >
Cornford, J. & Pollock, N. Theory and Practice of the Virtual University, 23 June
2000. Accessed on 15 Mar 2004.
< http://www.ariadne.ac.uk/issue24/virtual-universities/ >
Crawford, G., Rudy, J. A. and the EDUCAUSE Current Issues Committee. Fourth
Annual EDUCAUSE Survey Identifies Current IT Issues. Educause Quarterly, Vol.
26, No. 2., p. 12-26. 2002. Accessed on 26 Mar 2004.
< http://www.educause.edu/ir/library/pdf/eqm0322.pdf >
Creative Commons, Creative Commons. N.D. Accessed on 12 Mar 2004.
<http://creativecommons.org/ >
Crow, R. The case for institutional repositories: A SPARC Position Paper. The
Scholarly Publishing and Academic Resources Coalition. 2002. 15 Mar 2004.
< http://www.arl.org/sparc/IR/ir.html >
Cuban, L., Oversold and Underused, Harvard University Press, 2001.
CURVE, Curve Project Website, N.D., Accessed on 22 Mar 2004.
<http://iet.open.ac.uk/curve/>
Currier, S. and Campbell, L. M. Evaluating Learning Resources for Reusability: The
“DNER & Learning Objects” Study. Paper presented at ASCILITE 2002, Auckland.
<http://www.unitec.ac.nz/ascilite/proceedings/papers/059.pdf>
Currier. S, Barton. J, O’Beirne, R & Ryan, B. Quality Assurance for Digital Learning
Object Repositories: Issues for the Metadata Creation Process, 2003. Accessed on 15
Mar 2004.
< http://metadata.cetis.ac.uk/files/currbartobeiryan_altj_6.doc >
Czyborra, R. “Good Ole’ ASCII”. 24/05/1998. Accessed on 11 Mar 2004.
55

<http://web.archive.org/web/20030623064754/http://www.czyborra.com/charsets/iso
646.html >
Dalziel, J. Reflections on the COLIS (Collaborative Online Learning and Information
Systems) Demonstrator Project and the “Learning Object Lifecycle”. 2002. Accessed
on 15 Mar 2004.
< http://www.melcoe.mq.edu.au/documents/ASCILITEDalziel.rev.doc>
Darimont, A. W. Environmental Scan of Pricing Models for Online Content Report
II: Business Models for Object Repositories. 2002. Accessed on 15 Mar 2004.
< http://www.edusource.ca/craw/RPT_Pricing_Model205.htm >
Digital Bewaring Testbed. ,Migration: Context and Current Status, 2001. Accessed
on 5 Jan 2004.
< http://www.digitaleduurzaamheid.nl/bibliotheek/docs/Migration.pdf >
Dillenbourg, P. Virtual Learning Environments. Paper presented at the EUN
Conference, 2000. 2000. Accessed on 15 Dec 2003.
< http://tecfa.unige.ch/tecfa/publicat/dil-papers-2/Dil.7.5.18.pdf >
Downes, S. Design, Standards and Reusability. 2003. Accessed on 15 Dec 2003
< http://www.downes.ca/cgi-bin/website/view.cgi?
dbs=Article&key=1059622263&format=full >
Duncan, C. Talk given at The New Economy: Learning Objects, 2003. Accessed on
15 Mar 2004.
<http://dewey.intrallect.com/news/seminar_powerpoint/charles_duncan.ppt>
Dspace. Dspace: Durable Digital Repository. 2002. Accessed on 15 Mar 2004.
< http://libraries.mit.edu/dspace-mit/ >
EaseIT. Easeit Eng Evaluation Manual, N.D. Accessed on 15 Mar 2004.
< http://www.easeit-eng.ac.uk/main/manual.html >
Eduprise. An Introduction to Multimedia in WebCT. 2001. Accessed on 18 Dec 2003.
< http://www.immaculata.edu/OTS/IDRC/pdf_files/faculty/TipCT36_Multimedia.pdf
>
Emporia State University. WebCT Video Tutorials. N.D. Accessed on 11 Mar 2004.
< http://slim.emporia.edu/resource/WebCT/video_tutorials.htm >
EuroMET, Euromet, N.D. Accessed on 29 Mar 2004.
< http://euromet.meteo.fr/>
Fries, B. & Fries, M. The MP3 and Internet Audio Handbook. 2000. Teamcom.
Friesen, N. Three Objections to Learning Objects 2003. Accessed on 14 Mar 2004.
< http://phenom.educ.ualberta.ca/~nfriesen >
56

Friesen, N., Mourad, M., Robson, R et al Towards a Digital Rights Expression
Language Standard for Learning Technology. A Report of the IEEE Learning
Technology Standards Committee Digital Rights Expression Lanugate Study Group,
Presented at the Meeting of IEEE LTSC, Copenhagen, 9-11 December 2002,
Accessed on 15 Mar 2004.
<http://ltsc.ieee.org/meeting/200212/doc/DREL_White_paper.doc>
Gallagher, S. R. The New Landscape for Course Management Systems. 2003.
<http://www.educause.edu/ir/library/pdf/ecar_so/erb/ERB0310.pdf >
Glasner, J. JPEG Patent Claim Sparks Concern, Wired. 2002. Accessed on 18 Feb
2004.
< http://www.wired.com/news/business/0,1367,53981,00.html >
Greenstein, D. and Thorin, S. E. The Digital Library: A Biography. 2nd Edition.
2002.
< http://www.clir.org/pubs/abstract/pub109abst.html >
HaIRST, HaIRST project website. N.D., Accessed on 2nd
April 2004.
<http://hairst.cdlr.strath.ac.uk/about.htm>
Heslop, H., Davis, S. and Wilson, A. An Approach to the Preservation of Digital
Records. National Archives of Australia. 2002. Accessed on 22 Jan 2004.
< http://www.naa.gov.au/recordkeeping/er/digital_preservation/Green_Paper.pdf >
Higher Education Academy, Higher Education Academy. N.D. Accessed on 12 Mar
2004.
<http://www.heacademy.ac.uk/default.asp>
HLSI website, Accessed on 14Mar 2004.
<http://www.hlsi.org.uk/>,
IEEE LTSA. Draft Standard for Learning Technology — Learning Technology
Systems Architecture 2001. Accessed on 11 Mar 2004.
< http://ltsc.ieee.org/wg1/files/IEEE_1484_01_D09_LTSA.pdf >
IMS. IMS Abstract Framework: Whitepaper. ver. 1.0 , July 2003. Accessed on 11
Mar 2004 .
< http://www.imsglobal.org/af/afv1p0/imsafwhitepaperv1p0.html >
IMS. IMS Digital Repositories Specification. 2003a. Accessed on 13 Mar 2004.
< http://www.imsglobal.org/digitalrepositories/index.cfm >
IMS. IMS Question & Test Interoperability Specification 2004a. Accessed on 13 Mar
2004.
< http://www.imsglobal.org/question/index.cfm >
IMS. “Work in Progress”. 2004b. Accessed on 13 Mar 2004.
< http://www.imsglobal.org/workinprogress.cfm >
57

Institute of Educational Technology. Example of Versioning. N.D. Accessed on 15
Mar 2004.
< http://iet.open.ac.uk/curve/curvepublic2.cfm >
Ip, A., Morrison I. and Currie, M. What is a learning object, technically? 2001.
WebNet2001 Conference, Orlando, USA. Accessed on 11 June 2004.
< http://users.tpg.com.au/adslfrcf/lo/learningObject(WebNet2001).pdf >
James, H., Ruusalepp, R., Anderson, S. and Pinfield, S. Feasibility and Requirements
Study on Preservation of E-Prints. 2003. Accessed on 5 Jan 2004.
< http://www.jisc.ac.uk/uploaded_documents/e-prints_report_final.pdf >
JISC. Circular 1/01: Building MLEs across FE and HE 2001a. Accessed on 5 Jan
2004.
< http://www.jisc.ac.uk/index.cfm?name=circular_1_01 >
JISC. Circular 1/03: Managed Learning Environments for Lifelong Learning: Phase
2, 2001b. Accessed on 5 Mar 2004.
JISC. Circular 7/01: Digital Libraries and the Classroom. 2001c. Accessed on 5 Jan
2004.
JISC. MLE Interoperability Pilots in FE. N.D.(a) 5 Jan 2004. Accessed on 22 Mar
2004.
< http://www.jisc.ac.uk/index.cfm?name=programme_mle_pilots >
JISC. Building MLEs in Higher Education. N.D.(b) 5 Jan 2004.
< http://www.jisc.ac.uk/index.cfm?name=programme_mle_he >
JISC. Circular 7/02: Linking Digital Libraries with Virtual Learning Environments.
2002. Accessed on 5 Jan 2004.
<http://www.jisc.ac.uk/index.cfm?name=circular_07_02>
JISC. A Continuing Access and Digital Preservation Strategy for the Joint
Information Systems Committee (JISC) 2002-2005, N.D.(c). Accessed on 22 Mar
2004.
<http://www.jisc.ac.uk/index.cfm?name=pres_continuing>
JISC. Study of Records Lifecycle Structure (both FAM + RRS), by Elizabeth Parker,
N.D.(e). Accessed on 22 Mar 2004. <http://www.jisc.ac.uk/index.cfm?
name=srl_structure>
Jones, M. & Beagrie, N. “Preservation Management of Digital Materials”, 2002.
< http://www.dpconline.org/graphics/handbook/ >
58

JORUM. JORUM Scoping and Technical Appraisal Study: Volume I. Overview and
Recommendations. 2004. Accessed on 15 Mar 2004.
< http://www.jorum.ac.uk/vol1_fin.pdf >
JORUM. JORUM Scoping and Technical Appraisal Study: Volume III: Report on
Requirements Work. 2004c. Accessed on 15 Mar 2004.
JORUM. JORUM Scoping and Technical Appraisal Study: Volume IV: Appendices to
Report on Requirements work. 2004d. Accessed on 15 Mar 2004.
<http://www.jorum.ac.uk/vol4_fin.pdf>
JORUM. JORUM Scoping and Technical Appraisal Study: Volume V: Metadata.
2004e. Accessed on 15 Mar 2004.
JORUM, JORUM Scoping and Technical Appraisal Study: Volume VII. Digital
Rights Management (Draft)”, December 2003. Accessed on 15 Mar 2004.
JORUM, Project Document, Report on 5/99 JORUM+ questionnaire, McConnell, S.,
2003.
Katz, R. Balancing Technology and Tradition: The Example of Course Management
Systems. Educause Review, Vol. 38, No. 4, p. 48-59. 2003. Accessed on 15 Mar 2004.
< http://www.educause.edu/ir/library/pdf/erm0343.pdf >
Kenney, A. R. & Rieger, O. Y. Preserving Digital Assets: Cornell’s Digital Image
Collection Project. First Monda. vol.5 no. 6, June 2000b.
Korpela, J. A tutorial on character code issues. 20/02/2004. Accessed on 12 Mar
2004.
< http://www.cs.tut.fi/~jkorpela/chars.html >
Lamberson, M. & Lamb, B. Course Management Systems: Trapped Content Silos or
Sharing Platforms? 2003. Accessed on 15 Mar 2004.
< http://telr-research.osu.edu/learning_objects/documents/Lamberson-Lamb.pdf >
Laurillard, D. Towards a Unified E-Learning Strategy – Consultation Document,
Department of Further Education Studies. 2003. Accessed on 15 Mar 2004.
< http://www.alt.ac.uk/docs/ALT_tauels_response_final_20040127.pdf >
Learning Objects. Learning Objects for Introductory Programming, N.D. Accessed
on 29 Mar 2004.
< http://www.ics.ltsn.ac.uk/Learning_Objects/lmu_learningobjects/index.htm>
LeFurgy, W. G. PDF/A: Developing a File Format for Long-Term Preservation, RLG
Digi News. Accessed on 15 Dec 2003.
< http://www.rlg.org/preserv/diginews/v7_n6_feature1.html >
59

Libmng. The MNG reference library & related info. N.D. Accessed on 12 Mar 2004.
< http://www.libmng.com/ >
Linke, D. Creating TIFF files for electronic filing with the USPTO. N.D. Accessed
on 12 Mar 2004.
< http://www.dhdraw.com/help/tutorials/efs_tiff.htm >
Long, P. Learning Object Repositories, Digital Repositories, and the Reusable Life of
Course Content. Sylabus. 2004. Accessed on 11 June 2004.
< http://www.syllabus.com/article.asp?id=9258 >
LTPS, Learning and Teaching Portal Services. N.D. Accessed on 12 Mar 2004.
<http://www.ltsn.ac.uk/genericcentre/index.asp?id=18262& >
LTSC, “IEEE Learning Object Metadata Final 1484.12.1 LOM draft standard
document”, Learning Technology Standards Committee, 12 June 2002. Accessed on 1
Jan 2004.
< http://ltsc.ieee.org/doc/wg12/LOM_1484_12_1_v1_Final_Draft.pdf >
LTSN, BioImage. N.D. Accessed on 15 Mar 2004.
< http://bio.ltsn.ac.uk/imagebankuploads/Search/about.htm >
LTSN, 2002. < http://www.ltsn.ac.uk/ > Accessed on 12 Mar 2004.
Lupovici, C. & Masanès, J. (2000). NEDLIB Metadata for Long-term Preservation.
NedLib Report Series, no. 2. Accessed on May 3 2003.
<http://www.kb.nl/coop/nedlib/results/NEDLIBmetadata.pdf>
Lynch, C. A. Institutional Repositories: Essential Infrastructure for Scholarship in
the Digital Age. ARL Bimonthly Report, no. 226. 2003. Accessed on 15 Mar 2004.
< http://www.arl.org/newsltr/226/ir.html >
Martin, J. CTI (Computers in Teaching Initiative). Ariadne. no. 5. 1996. Accessed on
15 Mar 2004.
< http://www.ariadne.ac.uk/issue5/cti/ >
Mclean, N & Lynch, C. Interoperability between information and learning
environments – bridging the gaps, Draft version, IMS Global Learning Consortium &
Coalition for Networked Information, 28 June, 2003. Accessed on 15 Mar 2004.
< http://www.imsglobal.org/DLims_white_paper_publicdraft_1.pdf >
McCracken, R, Gilbart, M. Buying and Clearing Rights, print, broadcast and
multimedia ,1995, published by Blueprint, an imprint of Chapman & Hall, London.
MERLOT. Evaluation Criteria for MERLOT Learning Materials, 2000, Accessed on
15 Mar 2004.
< http://taste.merlot.org/projects/peer_review/criteria.php >
METS, METS: Metadata Encoding and Transmission Standard (2003a). Accessed on
1 May 2003.
60

<http://www.loc.gov/standards/mets/mets.xsd>
METS, METS:Implementation Registery. (2003b). Accessed on 14 Mar 2004.
< http://sunsite.berkeley.edu/mets/registry/>
MIT, MIT OpenCourseWare. N.D. Accessed on 12 Mar 2004.
<http://ocw.mit.edu/index.html>
Morrison, D. O’Leary, R. & Jordan, S. 3 VLE Strategies: VLE Strategies within three
UK Institutes of Higher Education N.D. Accessed on 10 Nov 2004.
< http://www.ltss.bris.ac.uk/vle02_3VLE_Strats.ppt >
Napier, Staff Guide: WebCT Content Module. 2003. Accessed on 18 Dec 2003.
< http://www.ed.napier.ac.uk/learntec/materials/Staff_WebCT_Content_Module.doc
>
National Library of Australia (1999). Preservation Metadata for Digital Collections.
Accessed on 3 May 2003.
< http://www.nla.gov.au/preserve/pmeta.html>
National Library of New Zealand. Metadata Standards Framework - Preservation
Metadata. 2002. Accessed on 25 Mar 2004.
< http://www.natlib.govt.nz/files/4initiatives_metaschema.pdf >
Nehrer, P. Castor Plug-in for Eclipse. 2003. Accessed on 12 Mar 2004.
< http://xdoclipse.sourceforge.net/index.html >
Nielson’s Jakob, heuristic evaluation.n.d. Accessed on 14 Mar 2003.
<http://www.useit.com/papers/heuristic/>
NEEDS, , n.d.Accessed on 22 Mar 2003.
<http://www.needs.org/needs/>
NISO. “NISO Data Dictionary - Technical Metadata for Digital Still Image”. 2002.
Accessed on 23 Dec 2003.
< http://www.niso.org/standards/resources/Z39_87_trial_use.pdf >
NLII, National Learning Infrastructure Initiative (NLII), Learning Objects (NLII
2002-2003 Key Themes 20 Nov 2003. Accessed on 15 Mar 2004.
< http://www.educause.edu/nlii/keythemes/LearningObjects.asp#nlii_research >
NLN National Learning Network. N.D. Accessed on 12 Mar 2004 from:
<http://www.nln.ac.uk/ >
NLN Paving the Way.n.d. Accessed on 14 Mar 2004.
<http://www.nln.ac.uk/materials/tutors/paving_the_way.asp >
Ockerbloom. Archiving and Preserving PDF Files. 2001. Accessed on 18 Dec 2003.
< http://www.rlg.org/preserv/diginews/diginews5-1.html#feature2 >
61

OCLC. Libraries and the Enhancement of E-Learning. 2003. OCLC E-Learning Task
Force. Accessed on 11 Mar 2004.
<http://www5.oclc.org/downloads/community/elearning.pdf>
OCLC [Online Computer Library Center] (2002). Preservation Metadata and the
OAIS Information Model: A Metadata Framework to Support the Preservation of
Digital Objects. OCLC/RLG Working Group on Preservation Metadata. Accessed on
3 May 2003.
<http://www.oclc.org/research/pmwg/pm_framework.pdf>
OKI. What is the Open Knowledge InitiativeTM? 2002. Accessed on 15 Jan 2004.
< http://web.mit.edu/oki/learn/whtpapers/OKI_white_paper_120902.pdf >
Olivier, B. & Liber,O. Reusing Online Resources, Kogan & Page, 2003.
Open Archives Initiative. Open Archives Initiative Frequently Asked Questions
(FAQ). 2002. Accessed on 13 Mar 2004.
<http://www.openarchives.org/documents/FAQ.html#What%20do%20you%20mean
%20by%20an%20"Archive >
Oren, T. Original CompuServe announcement about GIF patent. 1995. Accessed on
18 Dec 2003.
< http://lpf.ai.mit.edu/Patents/Gif/origCompuServe.html >
PADI. Roles and responsibilities. N.D. Accessed on 13 Mar 2004.
< http://www.nla.gov.au/padi/topics/8.html >
Paulsen, M. F. Online Education Systems: Discussion and Definition of Terms.
Accessed on July 2002. Accessed on 15 Dec 2003.
< http://www.nettskolen.com/forskning/Definition%20of%20Terms.pdf >
Perry, S. et-al. THE CO3 PROJECT: Stretching the IMS Specifications to Achieve
Interoperability. 2002. Accessed on 12 Mar 2004.
< http://www.jisc.ac.uk/uploaded_documents/CO3Report_Final.doc >
Pinfield, S. Managing electronic library services: current issues in UK higher
education institutions. Ariadne no. 29. 2001. Accessed on 14 Mar 2004.
< http://www.ariadne.ac.uk/issue29/pinfield >
Planet PDF. Planet PDF Tools List. 2003. Accessed on 12 Mar 2004.
< http://www.planetpdf.com/mainpage.asp?MenuID=193&WebPageID=612 >
Polsani, P. R. Use and Abuse of Reusable Learning Objects. Journal of Digital
Information. Vol. 3, No. 4., 2003. Accessed on 8 Jun 2004.
< http://jodi.ecs.soton.ac.uk/Articles/v03/i04/Polsani/ >
RLG, Research Libraries Group [RLG]. Trusted Digital Repositories: Attributes and
Responsibilities. An RLG-OCLC Report. 2002. Accessed on 3 Mar 2004.
< http://www.rlg.org/longterm/repositories.pdf >
62

Rinke, D. Creating TIFF files for electronic filing with the USPTO. N.D. Accessed
on 5 Mar 2004.
< http://www.dhdraw.com/help/tutorials/efs_tiff.htm >
Ritchie, J. The ScotCIT Programme – Achievements, Outcomes and
Recommendations, January 2002.
Ritchie, J. The Use of MANs Initiative - Achievements, Outcomes and
Recommendations. April 2000. Accessed on 25 Mar 2004.
< http://www.use-of-mans.ac.uk/umireport/noblanks.pdf >
Ritter, N. The Unofficial TIFF Home Page. 1997. Accessed on 12 Mar 2004.
< http://home.earthlink.net/~ritter/tiff/ >
Roelofs, G. Current Status of PNG. 7 Aug 2002. Accessed on 12 Mar 2004.
< http://www.libpng.org/pub/png/pngstatus.html >
Roelofs, G. Portable Network Graphics 2004. Accessed on 12 Mar 2004.
< http://www.libpng.org/pub/png/ >
Rogers, E.M. (1969). Diffusion of Innovations. The Free Press, New York,
Rogers, E.M. (1995). Diffusion of innovations (4th ed.). New York: The Free Press.
Santos, O & Ramos, F Computers and Education 42 (2004) p 227-242), 2004.
Sarrazin, P. Unisys/CompuServe GIF Controversy. 2003. Accessed on 18 Dec 2003.
< http://lpf.ai.mit.edu/Patents/Gif/Gif.html >
Sclater, N., Low, B. & Barr, N. Interoperability with CAA: Does it work in practice?
< http://www.lboro.ac.uk/service/ltd/flicaa/conf2002/pdfs/sclater_n1.pdf >
SCRAN website, N.D. Accessed on 14 Mar 2004.
<http://www.scran.ac.uk/freesite/subscribe/do-i-subscribe/>
SHERPA. SHERPA: Securing a Hybrid Environment for Research Preservation and
Access. N.D. Accessed on 15 Mar 2004. < http://www.sherpa.ac.uk/ >
SMETE. Federated Searching. 21 July, 2003. Accessed on 15 Mar 2004.
<http://www.smete.org/smete/?
path=/public/about_smete/activities/technology/federated_search/index.jhtml&xsl=1,
>
Smith, P., Place, E. and Price, S. Technical Architecture of the RDN Virtual Training
Suite: A National e-learning material for the UK. Institute for Learning and Research
Technology, University of Bristol. Accessed on 19 Mar 2004.
<http://www.vts.rdn.ac.uk/reports/ascilite02.html >
63

Stiles, M. Briefing Paper no 5. Pedagogy and Virtual Learning Environment (VLE)
Evaluation and Selection. n.d. Accessed on 15 Dec 2003.
< http://www.jisc.ac.uk/index.cfm?name=mle_briefings_5 >
Stiles, M. Staff Support and Sustainable eLearning. LTSN Generic Centre.
Sun Microsystems. Sun ONE Directory Server 5.2 Plug-In API Programming Guide.
< http://docs.sun.com/source/816-6702-10/overview.html >
TASI, Digital Preservation and Storage. n.d. Accessed on 23 Dec 2003.
< http://www.tasi.ac.uk/advice/delivering/digital.html >
The Social Informatics Research Unit, Education for Change Ltd & The Research
Partnership. Managed Learning Environment Activity in Further and Higher
Education in the UK. 2003. Accessed on 16 Jan 2004.
< http://www.jisc.ac.uk/uploaded_documents/mle-study-final-report.pdf >
Thorpe, M., Thorpe, K. & Kubiak, C. Reusing Online Resources, Kogan & Page, p
113, 2003.
Twigg, C. Improving Learning and Reducing Costs: Lessons Learned from Round of
the Pew Grant Program in Course Redesign. 2003. Accessed on 15 Mar 2004.
< http://www.center.rpi.edu/PewGrant/Rd1intro.html >
UCISA. “VLE Surveys – A Longitudinal Perspective between March 2001 and March
2003 for Higher Education in the United Kingdom”. 2003. Accessed on 15 Mar 2004.
< http://www.ucisa.ac.uk/groups/tlig/vle/index_html >
UMI Use of MANs Initiative. 2000. Accessed on 15 Mar 2004.
< http://www.use-of-mans.ac.uk/ >
Unisys. LZW Patent and Software Information. 2004. Accessed on 20 Feb 2004.
< http://www.unisys.com/about__unisys/lzw >
University of Leeds. Survey and assessment of sources of information on file formats
and software documentation. 2003. Accessed on 12 Feb 2004.
< http://www.jisc.ac.uk/uploaded_documents/FileFormatsreport.pdf >
University of Wollongong. Protecting your critical WebCT data. N.D. Accessed on
18 Dec 2003.
< http://www.uow.edu.au/lol/bkup_advice.html >
Whatis.com, Virtual Learning Environment. N.D. Accessed on 15 Dec 2003.
< http://whatis.techtarget.com/definition/0,,sid9_gci866691,00.html >
Wiley, D. The Instructional Use of Learning Objects (Online Version). 2000.
Accessed on 11 June 2004. < http://www.reusability.org/read/#1 >
64

Wiley, D, Learning Objects : Difficulties and Opportunities. N.D. Accessed on 14
Mar 2004. < http://wiley.ed.usu.edu/docs/lo_do.pdf >
Wilson, S. & Currier, S. What Is...IMS Content Packaging? 2002. Accessed on 11
Mar 2004.
< http://www.cetis.ac.uk/lib/media/CPbrief.pdf >
X4L, Rapid Resources. N.D.(a). Accessed on 5 Jan 2004.
< http://www.x4lrapid.info/resources.html >
X4L Healthier Nation. N.D.(b). Accessed on 22 Mar 2004.
< http://extranet.lauder.ac.uk/x4l >
X4l X4L website.n.d. Accessed on 2 Apr 2004.< http://www.x4l.org/>
Yee, R. and Beaubien, R. A Preliminary Crosswalk from METS to IMS Content
Packaging (2003). Accessed on 14 Mar 2004.
<http://raymondyee.net/files/2003/09/Library.Hi.Tech.latest.pdf>
65

Appendix A: Information on Projects
JORUM
A Scoping and Appraisal study was completed by the JORUM team in January 2004
with the view to informing the procurement of a JISC learning materials repository
system. The study involved an extensive study of end user requirements by making
use of focus groups, interviews and a questionnaire. Reports have also been written on
relevant technical information, metadata, digital rights management and accessibility.
The reports, in particular the end user studies, have been used to inform this study and
are publicly available from the JORUM website (JORUM, 2004a).
In addition, The JORUM project has provided support for the Exchange for Learning
programme (X4L, N.D.(c)) by making use of two trial repository systems (Intralibrary
from Intrallect and Xtensis from Xtensis e-Learning Ltd). This support will continue
until 2005.
From the report there are several possible functions for the JISC learning Materials
repository service. In the shorter term it is possible that the repository will be used to
hold learning materials produced through nationally funded projects. In addition it
may become a repository service for institutions where users can share their objects.
In the longer term, though certainly not at the first procurement phase, other functions
may be possible such as supporting collaborative working groups which may require
separate working areas, buying and selling of learning objects, and the long term
preservation of learning objects.
HLSI
The Higher Level Skills for Industry Repository (HLSI, (N.D.)) is funded by The
Yorkshire and Humber Regional Development Agency. It contains learning materials
to support the engineering/manufacturing sector and has currency across FE colleges,
HE and private companies. In March 2003, there were 12 partners involved. (Barker.E
& Ryan. B., 2003) . Resources are stored at three granularity level (object/asset, page,
document) and on 5 November 2003, the repository contained 8,000 objects with
metadata attached and 210 full learning packages.
NLN
The National Learning Network (NLN, N.D.) began in 1999. As a national
partnership programme funded by the LSC and other sector bodies the programme has
produced over 500 hrs of interactive learning materials, in the form of learning objects
which can be downloaded and used freely by FE institutions in England, Scotland and
Northern Ireland. Research is currently being undertaken into the requirements for a
repository to hold the materials.
66

COLEG
COLEG (COLEG, N.D.) is a self funding association of 42 colleges in Scotland. They
have been running since 1995 and have facilitated the sharing of resources between
colleges. Typically, colleges will contribute to a pool and receive peer-reviewed
resources at an exchange rate of 1:10. Their main focus has been on offline distance
learning materials, though they have been producing on-line materials in the past 3
years.
EuroMET
The EuroMET project began in 1995 and was funded by the EU to produce Computer
Based Learning Materials in Meteorology. The project involved 22 partners and the
lead organisation was Meteor Frances. The material was written in JAVA and consists
of 110 modules which each last for 45 minutes, on average. It is currently funded
under Eumetcal until mid 2004. EuroMET modules can be accessed from (EuroMET,
N.D.)
SCRAN
The Scottish Cultural Resource Access Network (SCRAN, N.D.) began in 1996..
They have an on-line database consisting of 1.2 million Objects (images, video, audio,
etc) and several thousand learning objects. They have adopted a long term
preservation strategy for the digital assets which they store.
Learning Objects for Introductory Programming
The Learning Objects for Introductory Programming (Learning Objects, N.D.)
modules began development in 2002. The work was carried out by staff at London
Metropolitan University in collaboration with the Bolton Institute and was supported
by the LTSN centre for Information and Computer Science. 54 Learning Objects were
created in total, 15 of which are multimedia resources featuring animations,
interactive models and self assessment quizzes.
67

Appendix B: Interview List for Long Term
Retention Project
Name
Current role Organisation
Michael Breaks University Librarian Heriot-Watt University
Joe Wilson Business Manager SFEU
Mary Macdonald COLEG Manager COLEG
Charles Duncan CEO Intrallect
Graeme Turnbull Publishing Manager SCRAN
Peter Douglas Consultancy Director Intrallect
Jean Ritchie Lecturer Edinburgh University
Balvier Notay Programme Manager JISC
Andrew Comrie Assistant Principal Lauder College
Kathy Wiles Senior Advisor LTSN
Sarah McConnell Learning Technologist JORUM
David Prescod Project Manager BECTA (NLN)
Moira Massey, Jackie
Carter
Project Managers JORUM
Sarah Porter, Steve Bailey Learning and Teaching
Programme Director,
Records Manager
JISC
Howard Noble Learning Technologist Oxford University
68

Appendix C: Focus Group
Focus Group Write-Up
Attendee Background
Sarah Currier CETIS
Lorna Campbell CETIS
Roger Rist Director of ICBL
John Casey Learning Technology
Sarah McConnell JORUM
Jenny Ure SCRAN – Educational Publishers
Peter Douglas Intrallect – Repository Vendor
Anne Hardie HE Lecturer
Tom McEwan Learning Technology
Gareth Knight AHDS
Hamish James AHDS
Ed Barker ICBL, Learning Technology
Tom Mcmaster FE – Library Resource Manager
Sarah Price JISC Regional Support
Wilma
Alexander
Learning Technology, Library
Michael Breaks University Librarian
Allison Cawsey HE Lecturer – computer science
Heather
Worlledge-
Andrew
University, Library
Nathan McIntosh ICBL, Learning Technology
Figure 1: Attendees at focus group
Acknowledgements
We would like to thank everyone for attending and contributing to the focus group. In
particular we would like to thank Peter Douglas of Intrallect for his help with note
taking.
Discussion Group A
John Casey University of Stirling
Roger Rist ICBL
Anne Hardie Napier University
Sarah Currier CETIS
Peter Douglas Intrallect
69

Session 1
THE GROUP WERE GIVEN INSTRUCTIONS TO CONSIDER WHAT ARE IMPORTANT
FACTORS FOR THE LONG TERM RETENTION AND REUSE OF E-LEARNING OBJECTS
AND MATERIALS DURING THE CREATION PHASE.
Key Issues
1 Format
2 Access Issues
3 Quality
4 Institutional
5 Media Design
6 IPR
7 Granularity
8 Metadata
9 Sense of Ownership
1
0
User Notes
Format
• It is important to consider what technical format the materials should have.
The effects of format on platform and download times are important
considerations.
• IMS content packaging and standards use,
Quality
• Quality was considered in quite a wide sense. All members of the group
considered this to be important
• Tutors need instructional design. Should training be available to help with this.
• Guidance notes for designers are also important
• The method of getting lecturers to produce materials should drive reflection
and training
Media Design Skills
• Media design includes a wide range of issues e.g Graphic Design, User
Interface, video, audio etc.
• Media design skills need to be available. It is not expected that the tutor will
be able to do all necessary media design.
Customisation
• Don’t want to have to go back to the original creators to make changes. It
should be straightforward to make alterations.
Sense of Ownership
• When lecturers create materials, it helps them to learn the subject themselves
and this improves the quality of teaching. This means that using others
material may not encourage teacher learning.
70

• Sharing is becoming necessary due to group size but it must be noted that
there is a lot of resistance to sharing.
Secondary Usage Metadata
• Secondary usage metadata is about resources associated with learning objects
and materials (I think). This would include things like evaluations, guidelines
for material use etc
Session 2
The aim of this discussion is to look at some problems in depth and try to find some
solutions. The basic procedure is to analyse the problem, write down what features the
solution might have and then form some example solutions.
Problem 1
Problem
Statement
Much has been said about the development of tools and
technology to make resource creation and reuse easier. There are
many options but what tools are the most necessary at the
moment.
Possible
Considerations
• Compliance with standards (Reload)
• Some sort of Unique Identifier and Resolver Service.
• What are areas, which tools could help with, that are not
being considered yet.
• There was general recognition that identifiers need to be there and handled
sensibly.
• Two tools were identified as being useful. The equivalent of BLOGS
communities for sharing learning objects and the Automatic Notification tool
for keeping validity of materials in check.
Problem 2
Problem
Statement
Quality checking/ Evaluating of resources is clearly an essential
process. What checks should be used? How do we make sure
they are fair and independent?
Possible
Considerations
• What sort of quality checks would be necessary for
institutional and national services?
• Who should do the checks/evaluations
• How should the results of evaluations made known?
• Peer Review
• Secondary Review
• Blogs- Connected Community. Blogs is an online system which allows
academic users to share knowledge about journals
• Automated Management
• Subject Area Groups – LTSN
• Evaluation – Independent, objective, reliable
71

• Evaluation should be done by the people that are going to use it
• Need to store information about how resource can be used, reviews etc
• There was agreement in the group that too many people are using the Amazon
comparison. It was not considered to be useful. Something like Blogs was
thought to be a more useful way for people to share resources and would be a
closer model to what happens in the real world (Meeting in
corridor/conference, you already know the person and trust their opinion, not
like anonymous amazon reviews) The group suggested that would like to see a
repository support this kind of communication/exchange of information about
resources.
• Automatic Management in a repository would be useful e.g flag up (email to
tutor) when a resource is, for example, 6 months old so tutor can go and check
to see if it is still valid
• National repository should include – Curriculum = core + bolt-ons
Problem 3
Problem
Statement
With the introduction of new technology, there is often poor
uptake due to staff having lack of time or incentives. What
should be done about this?
Possible
Considerations
• What incentives do staff need to share resources?
• How can staff be encouraged to use technology?
• How can time be made available for
training/experimentation? How can this be rewarded?
• We have poor uptake of learning resources
• Main issues are staff time, usefulness, dissemination, peer review,
customisation, metadata QA, Sense of ownership, user notes (secondary
metadata, information literacy)
• Dissemination – this has to be done for a long time and be planned
• Resources should be “permanent”, “managed” and “quality assured”. These
are library terms and all would be necessary for people to use materials and
resource and become confident in something like a national repository.
Incentives for Sharing
• Should they be like research journals which are peer reviewed and published.
This can lead to “Esteem”, “Prestige” and “Rewards”.
• There is a big difference between research and development.
• Maybe there should also be royalties
• The move to sharing culture needs to be carefully supported
• Should be recognised in thing like TQA
Encouraging use of technology
• Thee will need to be a change in the way of working to support Reuse
• Needs to be appropriate professional development and support services.
• Need to minimise staff time and maximise support. Funding is necessary
• Have to address both student and staff needs
72

• With learning materials projects we need to be clear about purpose and how it
will be useful. Projects in the past have not always addressed this.
Information Literacy
• Information literacy should be improved in students (who will eventually
become teachers) by teaching it in their degrees. This will include Learning
Resource Literacy.
• Information literacy should result in learning resource literacy.
Lack of access
• It should be mandatory to put learning objects into some form of safe and
accessible storage if the funding comes from JISC (for example). Edinburgh
University is following this kind of philosophy with a new group of projects
they are funding internally – all work needs to go into Edinburgh Universities
Repository and this was made clear when proposals for funding were invited.
• Meta-Collections – need to be able to search a network of repositories through
one interface.
73

Discussion Group B
Tom McMaster Lauder College
Tom McEwan Napier University
Wilma Alexander University of Edinburgh
Sarah McConnell EDINA
Lorna Campbell CETIS
Gareth Knight AHDS
Nathan McIntosh ICBL
This write up is based on notes taken by Gareth Knight and Nathan McIntosh
Session 1
THE GROUP WERE GIVEN INSTRUCTIONS TO CONSIDER WHAT SORT OF RESOURCE
STORAGE ISSUES WOULD HAVE AN INFLUENCE ON THE LONG TERM RETENTION
AND REUSE OF E-LEARNING OBJECTS AND MATERIALS.
KEY ISSUES
1 STORAGE
2 VERSION CONTROL
3 WORKING PRACTICE
4 IDENTIFIERS
5 PHYSICAL STORAGE
6 MAINTENANCE
CONTROL
7 BUSINESS MODEL
RESOURCE STORAGE
• RESOURCE STORAGE IS NOT THE MAIN PROBLEM. RETRIEVAL IS A MORE
DIFFICULT TASK.
• PEOPLE TEND TO EXPECT RETRIEVABILITY WHEN USING VLE’S AND THEN
DISCOVER THEY CANNOT RETRIEVE LOS
• “MANAGEMENT” METADATA IS NECESSARY TO IMPROVE RETRIEVAL
VERSION CONTROL
• NEED SUPPORT FOR DIFFERENT VERSIONS OF SAME OBJECT. THIS IS
BECAUSE CHANGES MUST BE TRACKED.
• NEED DIFFERENT METADATA TAGGING IF DIFFERENT PEOPLE USING SAME
OBJECT FOR DIFFERENT PURPOSES.
• AGAIN METADATA “MANAGEMENT” IS NECESSARY HERE
RESPONSIBILITY
• Is the creator responsible for use and reuse of learning objects, even if they do
not provide any guarantees? Is the creator liable for the content of learning
objects?
Scalability
• Issue of handling different versions of learning objects
• Storage space does not appear to be problem. Main issue appears to be
deciding how much information to store with resources
74

Access to Objects/Repository
• Who are the guardians?
• Who takes charge of allowing objects to be “checked in/out”
• Possible scenario is the lecturer/resource providers provide content to
facilitators who provide access to repository.
• Who pays for the facilitators? Institutions Would they pay for this type of
support?
• Librarians are presented as contolling access but they need to improve accesss
to content
• Athough the administration role is criticised no one wishes to be responsible
for storing reusable content.
• Need to find a way to communicate between librarians, technical people and
conceptual framework
Institutional Change
• Need for institutional change
• At the moment there is no business model in place to support the repository
we are discussing.
• “Craft Based” Notion limits the release/participation in creating/administrating
online learning objects.
• Creators are very resistant to allowing use of objects outside their sphere
• Publishing of papers solves incentive problem. How can this be adapted for re-
use of learning objects
• Need to provide incentives to add to repository and certainly to reuse them in
the future
1. Issue of having to “learn” someone else’s work if its quite complex
2. Breaking down resources into smaller “chunks” brings the issue of
granularity
Standards
• “Standards will support change, but cultural changes will require people to
alter their views of the world.”
• There is a need for greater understanding of standards and guidelines for
content management to assist in cultural change.
• Lauder College is currently working on standards but need support and
guidance. FE in general want guidance
Quality Control
o How do we ensure quality of addition to repository?
o Peer System like Amazon?
o Some institutions have QC on resources
o Who maintains the QC? Students? Lectures? Peers? Facilitators?
75

Session 2
Problem 1
Problem
Statement
E-Learning Objects and materials often contain links to web-
sites which may dramatically change or vanish. What
mechanisms need to be put in place to help with this problem?
Possible
Considerations
• Consider the individual user and the implications for
learning object storage in repositories.
• No-one designs for reuse. It is only considered later for (possibly) selfish
reasons.
• Argument against reuse of content – If its worth something, why make it
free?
• Something that is designed to be reusable should have less links
• Something that is for local use with correct support is likely to have more
links
• The rational needs to be found for using the link in the first place
• Method of “finding resources” as a learning objective in some cases
circumvents the problem.
• Web addresses are identifiers and not learning resources
• Asking questions of lecturers is needed such as:-
1. Why use the links in the first place?
2. Are copyright problems the issue?
• Groups Answer –Unless you minimise the links to external content there is
little that you can do. It is not possible to re-edit older content by locating
external links. Two people in the group commented that web-sites were not
resources, they were locations where resources were held. Therefore the issue
was not a great concern.
Problem 2
Problem
Statement
There is evidence that a lot of resources which are expensive to
produce are being lost. What systems need to be in place to
make sure that this does not happen?
Possible
Considerations
• Who is responsible for the resources?
• How can resources which are in danger of being lost
be acquired?
• Project Management should provide method of continuation from
inception
• Problems of opportunistic research projects not being planned to be
sustainable
• Funding Problem? Funding = guaranteed continuation + growth
• No one is responsible for the resource once the project has concluded
• Institutions are not set up to handle the requirements that the private sector
provide in this area
76

• Publishers “have to” make sure they keep systems sustainable
• Research Archiving?? JISC related/funded initiative
Problem 3: What features does a learning environment require to encourage reuse
• Relevant tools must be simplistic, intuitive (ie wizards) and provide inbuilt
interoperability. Returning to issue of cultural change, effort must be made to
train people in their use.
• Authoring tools RELOAD, authorware, etc
• Metadata editor
• Embedded naviagation (harder to reuse) vs Content Management Systems
(easier to use but need more work).
• Two schools of thought: One re-invents the material and the other makes no
changes
• Incentive based re-use strategy.
1. Rewarding for reuse
2. Increased speed for delivery of new course
3. “JIT” delivery of courses is a big requirement for FE
4. Removal of value judging FE/HE material by lecturers.
General
• Need to form a prestige economy. At the moment teachers rewarded for
research and not teaching
• A significant amount of content can not be reused because it is to focussed
• Current learning object have taken bad learning practices and solidified. At the
moment they do not encourage group work.
• A possible assessment method is to measure online support
• Should measure performance of teachers by number of student passes rather
than contact hours – indicates how culture change can be implemented.
• Problem of inherited work in modules and re/learning resources
77

Discussion Group C
Michael Breaks Heriot-Watt University
Jenny Ure SCRAN
Allison Cawsey Heriot-Watt University
Sarah Price JISC RSC
Hamish James AHDS
Ed Barker ICBL
Heather Worlledge-Andrew Glasgow University
Session 1
The aim of this discussion group is to look at what happens to learning objects and
materials after they have been stored. How should they be maintained and updated?
How can we ensure that the people who would find the resources useful are actually
aware of the resources and have access to them.
Key Issues
1 Metadata
2 Access and Ownership
3 Publicity
4 Digital Migration
5 Types of Maintenance
6 Users requirements for reuse
7 Commercial aspects
For courses closely linked to text books, it tends to be students that individually take
up related website materials etc.
Metadata
• Original creation of metadata  how does it support the long-term?
• Keeping metadata in sync with updates to the resource
• Store old versions of metadata
• Difficulty of transferring knowledge underlying the metadata from creator to
resource custodian such as librarian
• Content authors are needed for detailed metadata, but it comes with
assumptions about the audiences’ knowledge that may limit reusability
• However, paper resources have increasingly cross-disciplinary metadata, so
why not electronic resources also
Access and ownership
• Ethics (who gave what permissions in the original)
• It is often unclear about who owns the material. There is an accepted practice
which is used by lecturers. This is very different to books and journals where
there are clear conventions for use.
• Management of resources is difficult. There needs to be a gatekeeper
• Staff need training which should be done in synergy with other staff
development
78

• There are very different user groups. Some want large modules whereas others
would prefer to have small objects. We need to cater for both markets.
• Resources can get “worn out” if they are too publicised.
• Lack of credit and accreditation of original content creators in derivative
works
• Failure to seek permission, need to specify what is acceptable and what is not
• Are learning materials treated differently to publications?
• Seen as income generator, provide an institutional competitive advantage
o Restrictions on who can view and use resources, even within
institutions
o Opposite pressure to RAE (‘publish or perish’) as keeping e-learning
resources restricted may provide an income stream
• Need for enforcement or change of culture to allow reuse
o NLN free to use, CLIVE Vetinary science material sold at cost
o Whose budget are e-learning resources paid for from? Glasgow has a
core materials fund, but generally there is no central budget for e-
learning resources
o RAE style incentives?
Promotion and uptake of e-learning resources
• Need to promote e-learning materials to broaden usage, keeping in mind
maintenance and on-going support – not just initial effort
• Information overload and traditionalism act against the use of e-learning
resources, but there is a culture shift towards the use of more electronic
resources in learning and teaching [HJ: future sustainability issues?]
• No equivalent to publishing house publicity budgets exists
• Institutions lack a well-defined (and understood by staff) structure to support
the use of e-learning materials [HJ: technical support, password management,
subject knowledge etc may all be spread across different parts of the
organisations]
• Need to correctly target publicity
• Need to educate students and lecturers about good search strategies for finding
educational material
• Perhaps, e-books should be adopted as the best way of delivery e-learning
materials  familiar concept with advantages of electronic medium
o Could query the rationale of e-learning objects: it is a core
responsibility of the lecturer to create and customise learning materials
for their students  should they be using prepackaged learning objects
at all?
• HE has a craft based industry model
• Secondary user notes are added to original materials (in addition to metadata)
79

Session 2
2nd Breakout
Problem 1
Problem
Statement
Current theory suggests that learning objects can be combined
together in a variety of different ways to produce new learning
objects. This means it is difficult to track learning object use.
How can digital rights be ensured in such a way that there is no
‘visible’ cost to the end user?
Possible
Considerations
• What should be done currently? And what is possible for
the future
• Consider national storage and institutional storage.
Problem 2
Problem
Statement
Even if we have a “high quality” resource it is very difficult to
get widespread use in the academic community. How should
this problem be addressed?
Possible
Considerations
• Why are resources not being used?
• Do we know enough about what staff and students
really want and need?
• Who should be looking into this problem?
DISCUSSING RETENTION OF E-LEARNING MATERIALS:
• Main criteria should be actual usage and archival value (value for posterity)
• Timing of retention decisions in relation to the frequency of technological
change (as technological changes will trigger need to perform preservation
actions)
o Split decisions at the technical level from decisions at the use/subject
level
• There are high expectations when talking about retention of electronic material
• There is value in the ability to constantly update, change and correct and
electronic resource – this is not possible with a paper resource, but at the same
time some degree of version control and stability must be imposed
• What is different about e-learning materials? They are global and generic.
They lack a local context in which they are used (local expertise to inform use,
judge quality etc.); easily customisable, can lose provenance very quickly as
resource mutates.
• Distinction between HE and FE. FE has a much more structured curriculum
with defined outputs  standardised learning materials much easier to make
use of than in HE
o FE has higher contact hours, makes use of standardised materials
easier
80

o HE has individualistic approach  need to be able to customise
materials
• Need for ‘half-digested’ learning e-learning objects: quick access, consistent
metadata and quality
• Need critical mass to encourage usage
Charging/Use/dissemination
• Pay-per-click (micro payment) one way of funding development and judging
usefulness
• Create some type of market (credits for contributing material)
• Institutional attitudes, plans for income from materials, may inhibit wider use
and reuse
• Need to fund a study over 4-5 years to investigate what incentives work
• Subject based limits to dissemination may exist
• More centralised  easier to guarantee quality. Preservation easier with
centalised repository. Alternatively could have websites maintained by
individuals.
• Conflict between need to maintain and need to reduce intervention
• Users have far greater confidence in books than electronic materials  Really
need to sort out problems with quality checks.
• The group seemed keen on access stats to tell how useful a resource is –
Cheap and effective
• Peer Review methodology
• Should customisations of resources be stored? Or do we just need standard
packages?
• There is a lot of mistrust in the academic community – why is this being done?
Is it a threat to lecturers jobs
• OAI for cross searching is crucial
• Need to allow time for critical mass to form when setting up collections. Need
4-5 year period and heavy investment. Piloting in narrowly defined subjects
can be useful
• Methods of checking quality – usage, supervised usage, consultation with
academic community, author reputation
General Discussion Points
• Short-term funding  long-term uncertainty
• Blogs, Amazon reviews, the Google search paradigm  relevance to
encouraging the use of e-learning materials?
• Dawinian model of e-learning – Just leave it alone and see what survives
o Focus on patterns of reuse
• Notion of publishing can be introduced to help control versions, establish
rights,
o Gatekeepers for repositories
• E-Learning materials need quality assessment
81

• E-Learning objects have fuzzy boundaries. This is a very different situation to
paper based materials
• For the case studies which we are considering for this study, pedagogy may
not be in the objects. Really need to speak to users. Was the material actually
reused
• !0 million unique identifiers required – Who pays, whomanages
• Teachers roles are changing – from creating learning content to being context
providers
82

Appendix D: Analysis of Interviews, Focus
Groups and JORUM study
This chapter reports on recent studies undertaken to determine the current concerns
and thoughts of the E-Learning community. This includes information from the
interviews conducted during this study (Appendix B) and the focus group carried out
as part of this study (Appendix C). In addition the JORUM (JORUM, 2004) has
recently published a report on requirements gathering for a national repository. This
included several telephone interviews, focus groups and a questionnaire. In order to
distinguish betwee the studies done by this project and JORUM, studies by this
project will be referred to as LTR.
It should be noted that for both the studies conducted by JORUM and this project, the
majority of participants were people with a strong interest in the field. This may mean
that some of the results do not reflect the views of the whole academic community.
In addition the issues involved in this study are complex since there are so many
factors involved. There is the issue of timing since the field changes very quickly with
technological advancements; scalability as some people are thinking of national or
widescale use of e-learning materials whilst others are thinking about ones used in
institutions; the problem of unclarity about what learning objects are – from word
files to simulations. This can make it difficult to determine exactly what is meant by
some comments.
General Attitudes to E-Learning
There is an issue of trust in the academic community about E-Learning. Several
statements have been made during the interviews and focus groups including:
• “Hype is often necessary to get funding for e-learning projects”
• “Digital resources can mutate and lose provenance quickly”
• “E-Learning Objects have fuzzy boundaries unlike paper resources”
• “Communities can not agree on definitions for E-Learning concepts”
• “Users have far greater confidence in books than learning materials”
In the JORUM questionnaire there were also concerns about use of e-learning
materials
• “E-Learning projects only used by small subset of the community“
• “I am a library systems manager and I can see little demand for this kind of
thing. Even E-Journals get very little use and a lukewarm reception from even
the most enthusiastic on-line individuals despite years of trying to promote
such products. I find it easier to help people find books and journals that they
can pick up and read with no barriers, password, inductions etc”
83

However many people interviewed have stated that there is a definite move towards
the use of digital resources. This has been commented on with respect to E-Journals,
VLEs and repositories.
• “Just about every FE college in Scotland has a VLE strategy”
• “There is a move towards VLEs in Universities”
• E-Journals are being used more
This move was thought to be motivated by easier storage and retrieval. In particular,
the relatively recent concept of digital learning objects has brought much enthusiasm
to the e-learning community. Most of the people interviewed thought that there is
clear potential which should be realised. The main advantages seen are
• Saving Time by reducing duplication of effort
• Improving quality
However there is some scepticism about how useful e-learning objects are as the
following comments show:
• Theory has not been clearly defined yet and there is no clear message
• “Learning objects have limitations and may not be useful for every subject”
• “Repository of Learning materials is unrealistic at object level due to technical
problems and Licensing issues”
• “It is a core responsibility of the lecturer to create and customise materials”
• “Assumption that if one person finds a resource useful others will”
It was also interesting to note that granularity came up as several people in the
JORUM questionnaires thought that digital assets had more potential than E-Learning
Objects whereas the LTR focus group had the idea that “Perhaps e-books should be
adopted as best way of delivering e-learning materials”.
There is much uncertainty in the use and value of E-Learning Objects. In particular
many people LTR interviewed and in the LTR focus group were concerned that no
business model has been made for learning objects yet
Everyone seemed to agree that a lot of change would be necessary in order to get E-
Learning Objects adopted.
• Teaching and Learning is a “craft based” industry
• The “not invented here” problem is difficult to overcome
• People need to learn from someone elses work and that can be time consuming
• There are many people interested in sharing but others interested in profit
which leads to IPR problems. More clarity is necessary
• One person at the focus group commented that “At the moment Learning
Objects have taken bad learning practices and solidified – do not encourage
group work.”
• “Teachers roles are changing from creating learning content to being context
providers”
However there is disagreement over what stage we are currently at.
Quality Assurance
84

Many people stated that quality assurance is essential in order for people to use the
learning objects. However it was also noted from the LTR focus group and interviews
that
• It is very difficult to judge quality
• It is very difficult to judge the impact of learning objects and materials
• It is difficult to know how resources are used
• The pedagogy will not always be in the object
One person interviewed felt that there should be no judgement on quality and that
good resources would survive and bad ones would not get used.
On a more practical level, “It is not clear who should be involved in Quality checking:
lecturers, students, peers or facilitators”. A number of methods were suggested
including:
a) Peer systems like Amazon
b) Blogs type community of practice
c) Use of access stats
d) Supervised usage
e) Consultation with academic community
f) Author reputation
There was a lot of uncertainty in which methods should be used but it was noted that
cost and how people would actually use the review methods needs further
investigation
Sharing of Resources
There is a lot of debate about the value of learning objects and few commercial
companies are investing in resource creation for the FE and HE sectors. In the
JORUM questionnaire, the effect of the MIT OpenCourseWare (MIT N.D.)initiative
was mentioned and how it may have an influence in encouraging institutions to make
their materials available free of charge. The creative commons licensing scheme
which allows materials to be used free of charge for non commercial purposes is
recognised by many as an important step.
Many people in the e-learning community are talking about sharing materials, some
as though it is about to happen on a large scale and others as though it is a nice idea
but only likely to occur between a few enthusiasts.
Some viewpoints are that:
• Creative commons would have problems in that a) no incentive for
maintenance of resources b) problems of plagiarism c) difficult to police for
commercial use.
• “If its worth something why make it free”
• “Institutions will keep the good stuff for themselves and put the dross online”
• Creators are often resistant to objects use out of sphere
85

The JORUM questionnaire and focus groups have looked at peoples attitudes towards
sharing materials in the context of a national repository. The comments suggest that
several people think that this is a critical period for the sharing of resources and that
decisions made now may affect the scale of sharing in the future.
• “The move to a sharing community needs to be carefully supported”
• “Having learning objects for sale could prove fatal to the sharing ideology.”
• “Do we want to follow overpriced text book model?”
• In the JORUM questionnaire 79% of people who responded were interested in
sharing learning objects
Although the vast majority of people who answered the JORUM questionnaire were
in favour of a national repository, there was some doubt about how sharing of
resources would occur.
Many of the people interviewed by the LTR study suggested that a distributed
approach to repositories which can be linked together may be the most appropriate
option. A point made as a JORUM questionnaire response was
“Need to start looking at an institutional and regional sharing before we go National.
It is hard enough to create a sharing community locally, let alone nationally”
Models of sharing are being discussed on institutional and national levels. For
institutions it has been noted from the JORUM questionnaire and Focus Groups that:-
• FE colleges often use shared folders
• Institutions may be competitive and not want to share with outside institutions
• FE Colleges have formed consortia for sharing materials (mostly on a regional
basis)
• HE tend to share more on a subject basis
• Institutions could allow others to search their repository by OAI
Many people in the Learning Technology sector are also talking about Peer to peer
file sharing like Napster e.g. Lionshare which can facilitate sharing between closed
communities.
There were a number of people from the JORUM questionnaire and the interviews
who questioned the incentive for the individual to make resources freely available
“What is the incentive for the individual – more work and no gain”. It was thought
that Individual and institutional incentives are both necessary and many comments
were made in the LTR and JORUM studies along the lines of
• Possibility of a “prestige” economy
• Rewards such as money or credits to learning teaching CV for individuals
• RAE style rewards – similar to research
However it was pointed out that currently the research “publish or perish” mentality is
the opposite to Learning materials in which there is more pressure not to share them.
Other possibilities to develop sharing included rewards for institutions in gaining
resources through consortia approaches. One interviewee commented that consortia
agreements have the problem of having to rely on other people producing good
quality materials on time.
86

Resource Retrieval and Reuse
Dissemination and use
One point which many interviewees agreed with was that even if learning objects are
good, it is incredibly difficult to get people to use them. This was considered to be one
of the most difficult problems
• “Even if learning objects are good people will not use them”
• “Dissemination is not that easy – communication between staff is not always
good. Different dissemination mechanisms are necessary in different
institutions”
• “The real problem is letting people know and keeping awareness up”
• “Need to promote e-learning materials to broaden usage keeping in mind
maintenance and ongoing support”
• “Responsibility for resource awareness is a big problem”
It was also noted that:
• Dissemination needs to be done over a long period of time and be planned
• The audience needs to be targeted correctly
• There is no equivalent to the publishing house budget
Many people also commented on problem of resources being promoted before they
are good enough. There were also many comments made about how librarians
currently tend to control access and that they may have to be more involved in
promoting access.
For use,
• Staff have to gain something
• Staff do not have much time to search
There were also many people who felt that lecturers would like to modify resources
however one response was that “lecturers rarely modify materials in practice”. It was
also commented that by many that most users would like to customise materials/
Also to use resources and collections a certain amount of skill is required and many
people in the LTR focus groups and interviews commented on the need for staff
training and information literacy.
There was uncertainty in how resources were used. For example some people thought
that NLN resources were widely used whereas others felt that they were hardly used
at all. One learning technologist suggested that it is important to try and get some
level of awareness about how much resources are being used.
It was thought that the way the end user receives so they can contextualise the
resources in order to use them is also significant. One comment from an interviewee
was
• “Learning objects should be seen as being like a string of pearls
Strategy for Formation of Collections
87

Current concerns about resource retrieval include
• “Information overload”
• “People do not know where to find appropriate electronic learning materials”
• “Too many different places you can go to find things”
Most people would agree that a strategy for dealing with this information is important.
The FE and HE community also consists of many diverse groups and they have many
different requirements. Individual differences will exist between different institutions
however in general people though that:
• It is more easy to dictate institutional strategy in FE
• FE have less money so only one VLE or ILT strategy
• Curriculum is very important in FE
• FE cater more for market needs and produce “Just In Time Courses”
• FE are less likely to need to customise materials than HE
• A major barrier for FE is not enough available technology
• In HE different departments do things in different ways and may have
different IT and resource policies. New Universities are more likely to be
centrally managed than old ones.
• HE communities are more subject based whereas FE communities are more
region based.
• HE has strong research focus whereas FE has only teaching focus
However, there were thought to be important similarities between FE and HE in thiat:
• For both HE and FE, the number of students is critical.
• The principal will make the strategic decisions about resource management
and ILT.
There were also comments made about the different requirements of different user
groups within FE and HE:
• “Different communities of practice have different requirements”
• One interviewee noted that subjects such as engineering and medicine tend to
have more interest in computer based learning.
This viewpoint was echoed by some people in the JORUM questionnaire responses:-
“Medicine would need areas that others can not see for ethical reasons” - JORUM
Many people commented on the issue of centralised collections. There were several
advantages to centralised collections mentioned during the LTR and JORUM
surveys:-
• More centralised makes quality guarantee and preservation easier
• Smaller Universities and colleges may prefer to use central system
One response in the JORUM questionnaire suggested that:
• Centralised collection could be split into subject areas
And an interviewee from LTR stressed that:
• Quality is essential for centralised collections
88

Although most people in the JORUM questionnaire were in favour of a national
repository (89%), there were some doubts about how a centralised collection would
work in practice.
• May only be used by a small fraction of FE and HE community
• One comment in the JORUM questionnaire stated “Many companies have
given up on the big database idea – no one uses, no one contributes, hard to
keep current, costs money and quintessentially uninspiring to use”
The other approach would be a more distributed system and many of the people
interviewed for the LTR felt that a distributed network may be more appropriate.
• In practice distributed is more likely to work than centralised systems
• Building at local level and joining is maybe easier
• The HLSI have a regional and subject model and intend to expand to other
regions
The people interviewed also commented on linking of collections
• “Need to be collection at all levels from consortia, national scale and
individual college repositories. These collections need to be linked.”
For both types of collections, the formation process was thought to be important. In
particular
• Resources can get worn out if overpublicised.
• Many people commented in time being necessary to form a critical mass
• Once critical mass of usable materials is achieved should be easier to get users
There is also the issue about how new collections which are to be set up would work
with other collections. There was also concern in the JORUM questionnaire and
Focus groups about reinventing the wheel.
• “LTSN and NLN do some of the same stuff”
• Similarities with Ferl, UkEU, UfI etc were also commented on
• “How does a centralised repository fit with others like fenc and college
online”
This leads to issues about duplication of effort and how to best link collections.
If something is to be done about retaining and reusing resources for a long period of
time it is necessary to think about the end users and the functional requirements. This
requires a strategic approach though it has been noted that the best methods may not
always be used for political or economic reasons.
The different functionalities of collections mean that important consideration needs to
be placed over what information needs to be stored with resources. In the focus
groups and interviews several issues kept coming up:
• Metadata
• Reviews or quality assessments
• Licences
The creation of metadata for Learning Objects was considered to be an area which
requires further investigation during the focus group. Methodologies for metadata
creation came up in both the LTR and the JORUM studies:
89

• “Content authors needed for detailed metadata but assumptions made about
audiences knowledge. Paper resources do have increasingly cross-disciplinary
metadata”
• Different metadata tagging is necessary for the same object if using for
different purposes”
• “Metadata methodology is developing and we will have to modify
approaches”
• “Difficulty of transferring knowledge from the creator to the librarian”
In particular problems with classification of resources were noted, especially
educational classification. This was noted in the JORUM study by X4L projects
• Classification schemes are very difficult – major barrier
For FE, it was commented by many people on the importance of classification by
curriculum. However two interviewees thought that a lot of people pay too much
attention to curriculum classification and will ignore good quality resources.
The different functionality of storage facilities can make metadata retrieval a problem
The JORUM study looked into the issue of who should create metadata and how,
based on comments obtained during their survey.
• “Metadata is best written by subject experts – along with information
professionals”
• “6 Disparate people would enter different standards of metadata
• Some people felt that the author should also have some role in entering
metadata
• JORUM came up with three models Author, Author and Information
Professional, Information Professional
Management of materials was also seen as a key issue by many people
• “Version Control – metadata management is difficult
• The publishing process is also important for IPR and QA
• Decisions need to be made about what information to store with an object.
On a national level, it was felt by many people interviewed for the LTR study that
unique identifiers are important to assist with management of the materials.
• Identifiers need to be in place”
• “Two way flow between archive and delivery system would mean unique
identifiers are crucial”
• 10 million unique identifiers who pays” – focus group
Traditionally, IPR has been a big problem in the E-Learning sector and the issue came
up many times in both the LTR and JORUM studies
• Individuals want rewards”, “Institutions may see Los as assets”
• Uncertainty over ownership
• Huge problem with IPR
• IPR is the major issue
It was not clear how to solve the problem but some people had interesting views
90

• Smaller resource have less IPR
• Should concentrate on income generation once tech probs solved
Also many resources were only available for non-commercial use and one interviewee
commented on how it was unclear about what this meant and how to police for it.
In addition there are problems with modifications to materials and metadata
• Modifications to materials may require a high degree of skill
• Modifications will have to be made to metadata
• Are versions of the same object kept
• “Conflict between need to maintain and need to reduce intervention
All the dealings with versions, metadata, IPR etc requires a lot of management as
noted from the JORUM studies:
• “Noone wants responsibility for storage”
• “Who would pay for facilitators”
• “Facilitators need to provide access”
• “Who are guardians, who takes charge”
It was also noted by many in the JORUM and LTR studies that institutions may lack
the expertise to deal with the collections.
• “Institutions lack a well defined structure to support the use of E-Learning
Materials”
• “Some institutions have a lot better attitude towards resource management
than others”
• For JORUM it was commented that gatekeepers necessary though they slow
the process down.
• In libraries academics give lists to librarians of books that they need can this
be adapted
One of the JORUM focus groups commented on how keeping metadata and links was
far easier to manage than keeping the actual resources. It was also mentioned by many
of the JORUM study contributors that smaller digital assets rather than Los should be
the main focus as they are much easier to handle
Design for long term reuse
If designing materials for long term use then it is necessary to consider what is needed
for long term storage. People had various views about metadata for archiving
• Little metadata is needed for archiving
• How does original metadata support long term retention
• Need technical and rights metadata
From the LTR interviews, several technical aspects of design were considered
important long term reuse
• Use of appropriate file types
• Use of appropriate software so resource can be used on other systems
• Try to keep storage separate from delivery
• Try to keep all files necessary to use a learning object in one package
91

• Try to make sure that lesson is not embedded in the technology
• No standards exist for describing interactions which are not simulations
It was also considered important to design for customisation and to make necessary
IPR clearances at the design phase. In addition it is necessary to think about how the
resource will interact with current institutional systems. Comments from the
interviewees included:-
• There is a need for integration with college VLEs
• Large collections such as NLN could be drivers for standards use in VLEs
• Difficulties exist when trying to transfer packages to and from VLEs
• Want to be able to render things separately for new systems
• There needs to be a link between VLEs and repositories
It was acknowledged by many people that currently, there is not much design for
reuse and that it is not easy to get institutions to make the extra effort required.
Several comments were made during the LTR and JORUM surveys.
• Design for large scale reuse can be very expensive and often requires a team
effort
• Often difficult to get resource producers to follow style for reuse
• Media design skills are in short supply
• Tools for helping with design and metadata should be simplistic and have
inbuilt interoperability. Currently tools are typically quite difficult to use
• Specifications keep changing
It was thought that Documentation and guidance for resource design is necessary. In
addition the issue of incentive is important as authors don’t always get enough credit
for the work.
Views on Long Term Storage of Resources
The natural time which course material tends to be useful for before serious updating
is required was typically thought to be around 5 years. During this time alterations
may be made depending on the nature of the materials. After this time the materials
could still be useable but may be dated. Several people interviewed have explained
that the pedagogy behind the resource is likely to be valid after this time as will the
assets which make up the resource so new materials should not have to be re-written
from scratch.
One comment from the LTR focus group was that the main criteria for long term
retention should be actual usage of the resource and archival value.
The report of the JORUM focus groups noted that “Archiving of materials is a low
priority for the community and more research into digital preservation is necessary
before this should be considered”. It was noted during many of the LTR interviews
that there is low awareness of the issues involved in keeping resources over a long
period. In particular there was little evidence of long term strategies being used in
institutions.
92

Even though people are not aware of the need directly, some lecturers from the X4L
healthier nation project had problems with web-sites not being available or changing
which can be very disruptive to classes. It has also been noted that institutions are
beginning to look at repositories due to the potential for longer term storage. However
VLEs are also being used in such a way that is not necessarily geared up for long term
storage.
Many of the people interviewed by LTR acknowledged that useful information is
being lost from national funded projects in the UK.
• “There have been problems in the past and several projects have suffered from
materials and knowledge being lost”
• “If a project dies a lot is lost”
• “There is some urgency here. As well as funding new projects it is important
to keep old ones. Need to bring attention to the fact that resources are being
lost”
There were thought to be many reasons for this loss of resources including:
• No-one is responsible for a project once it has concluded
• “Project management should provide method of continuation from inception”
• “Opportunistic projects are not planned to be sustainable”
• “It should be mandatory to put learning objects into some form of safe and
accessible storage if funding is coming from JISC for example”
• “Short term funding implies long term uncertainty”
This does appear to be changing as a lot of 5/99 materials are being deposited with
other JISC services such as AHDS. Also most X4L projects are using JORUM for
depositing materials
It was also noted by several people that there have been technical problems with
keeping resources available for a long period. For example file-types and software
used may affect the reusability and the resource is likely to go out of date after a few
years.
One Learning Technologist thought that by saving resources from past projects,
• “This could help the e-learning community build on resources rather than the
stop, start cycle which has been happening”
However others have a different approach
• one interviewee commented that “Good resources should survive but bad
ones will be forgotten about”
• The “Darwinian” survival of the fittest approach
Others think that the infrastructure to allow the fittest resources to survive is not in
place yet. It was suggested that there should be “some way of finding resources that
need saving” and that more has to be done to tackle the problem of “awareness of
resources”
At an institutional level, it does not appear that there have been long term policies for
learning materials though institutions are utilizing VLEs and repositories to help
93

facilitate it. It was noted by one interviewee that - By 2005 there will be records of
achievement for students and that in the future there may well be more need to audit
trail. In particular the long term archiving of assessments was identified as being of
potential importance.
A librarian noted that “For long term storage, there is a definite cost. It is difficult to
see where this would be applied. One method would be the individual resource creator
pays. Another way would be for funding councils to give more money to make sure it
happens”
Another interviewee commented on the importance of location and funding for long
term storage
• “JISC could not hold materials for long period. It would have to be done
institutionally or by library”
• “Institutional archiving is possible. We could also have a centralised system
which people use as a service”
• British Library could be involved
Implications
From the studies it is clear that most of the people involved saw that there are
potential benefits for digital resources to be used more in teaching and learning in the
FE and HE sectors. The use of learning objects was debatable but this was partly due
to the different definitions used and the distinction between individual assets and
learning objects.
In the future a fundamental change to the way educational resources are managed and
used seems to be necessary to avoid resource loss and duplication of effort. In
particular the way resources are designed should be looked at along with how the
resources are managed once they have been created. All this will have a high initial
cost and require a lot of experimentation but should lead to benefits in the future.
Important barriers of IPR, shortage of skills in designing for reuse and the costs of
resource management mean that institutions should only be involved if they are aware
of the risks and uncertainties. Institutional efforts could be funded or partially funded
by appropriate funding councils in exchange for resources or evaluations of
effectiveness. To assist with this, evaluation studies should be undertaken to find out
how well current resources like NLN are used.
There is also a general agreement that good resources created by national money have
not been used as much as they could be. With resources that are currently being
created, conditions of funding could be used to ensure longer term retention. With
resources that have already been created it may be possible to fund modifications to
make them more suitable for reuse.
94

95

Long-Term Retention and Reuse of E-Learning Objects and Materials

More Related Content

What's hot

Similar to Long-Term Retention and Reuse of E-Learning Objects and Materials

More from Baker Khader Abdallah, PMP

Recently uploaded

Long-Term Retention and Reuse of E-Learning Objects and Materials