Pots pan final report

Document title: Jisc PotsPan Project Final Report
Last updated: May 2013
1
Project Document Cover Sheet
PotsPan Project Final Report
Project Information
Project Acronym PotsPan
Project Title Pilot of the SWANI Project Administrative Network
Start Date 1st
May 1012 End Date 30th
April 2013
Lead Institution Coleg Sir Gâr
Project Director Dave Howells
Project Manager &
contact details
Professor Tony Toole
tony.toole@e-college.ac
07964894790
Coleg Sir Gâr, Graig Campus, Llanelli SA15 4DN
Partner Institutions Pembrokeshire College, Swansea Metropolitan, The Prince’s
Trust, PRP Training, JISC RSC Wales
Project Web URL Project Website: http://potspan.pbworks.com/
Project Blog: http://potspan.blogspot.co.uk/
Programme Name
(and number)
JISC e-Learning Embedding Benefits
Programme Manager Rob Englebright
Document Name
Document
Title
Final Report
Reporting
Period
May 2012 – April 2013
Author(s)
& project
role
Tony Toole, Project Manager
Date March 2013 Filename PotsPan Final Report
URL http://potspan.pbworks.com/w/file/63770491/PotsPan%20Final%20Report.docx
Access
Document History
Version Date Comments
V1 19/02/2013 First Draft
V2 27/03/2013 Second Draft
V3 06/05/2013 Final Draft

JISC PotsPan Project Final Report
2
JISC Project Final Report
Project name: Pilot of the SWANI Project administrative network (PotsPan)
Author: Tony Toole
Contact: tony.toole@e-college.ac
Date: May 2013
Contents
Acknowledgements 2
1. Executive Summary 3
2. Background 4
3. Aims and Objectives 4
4. Methodology 4
5. Implementation 5
6. Outputs and Results 5
Case Study 1: Coleg Sir Gâr 6
Case Study 2: Swansea Metropolitan 7
Case Study 3: Pembrokeshire College 8
7. Outcomes 9
8. Conclusions and Recommendations 9
References and Appendices 10
Acknowledgements
The PotsPan Project was funded by JISC as part of the e-Learning Embedding Benefits
Programme. The project team would like to thank Rob Englebright, the Programme
Manager, and staff at RSC Wales for their support and assistance. We would also like to
thank all those in the WBL teams in the partner institutions and in particular, the excellent
work by Lewis Jones who built the digitised WBL administration system.

3
1. Executive Summary
The PotsPan (Pilot of the Swani Project administrative network) project was funded as part
of the Jisc e-Learning Embedding Benefits Programme. The goal was to take the outcomes
of the successful Swani (Secure Work-based learning through Networked Infrastructure)
concept-proving project completed by Coleg Sir Gâr in 2012 and to pilot it more broadly
across the regional partnership.
The Jisc Swani Project created a digitised administrative and management documentation
system for Work Based Learning in Wales that included a digital signature system that
satisfied the audit requirements of the EU and the Welsh Government.
The PotsPan project piloted the system with two of the partners in the Skills Academy Wales
South Wales WBL consortium and also evaluated the approach in a partner University for
potential use in the administration of a new distance learning WBL programme.
The purpose of the PotsPan project was to take what was a successful concept proving
project and develop it into a practical digitised WBL administrative system that would not
only satisfy the needs of the SWASW partnership, but would also be of value to other
institutions addressing similar issues.
The broad aim of the PotsPan project was to implement the digitised online WBL
administrative system, created by the Swani project. The key objectives were to:
Refine, update and test the WBL document management system to ensure it met the
efficiency, usability and data consistency design requirements;
Test and evaluate the system with cohorts of work based learners in the SAWSW
partnership;
Explore the use of digital pens and electronic signatures as part of the WBL
programme audit arrangements;
Contribute to the European debate about digitised document management and the
use of electronic signatures for audit purposes.
The outcomes of the PotsPan project led the project team to conclude that the work had
confirmed the main benefits of administrative system digitisation:
Digitisation of the administrative system eliminated the need for the repeated entry of
core data and ensured data consistency;
Digitisation made the entire administrative system available to programme managers
at all times and from any location;
The system delivered significant operational efficiencies, mainly from the avoidance
of physically transporting learner files between different geographical locations.
The overall conclusion was that the PotsPan project had achieved its primary goal of
exploiting the benefits of the original Swani project and demonstrating its value and viability
in the practical administration and management of the work-based learning programme in
South West Wales. Furthermore, it was felt that the benefits were transferable to other
programmes with similar needs and hence contributed positively to the Jisc e-Learning
Programme Embedding Benefits objectives.

4
2. Background
The JISC SWANI Project1
successfully created a digitised administrative and management
documentation system for Work Based Learning in Wales that included a digital signature
system that satisfied the audit requirements of the EU and the Welsh Government.
The PotsPan project piloted the system with two of the partners in the Skills Academy Wales
South Wales WBL consortium and also evaluated the approach in a partner University for
potential use in the administration of a new distance learning WBL programme.
The purpose of the PotsPan project was to take what was a successful concept proving
project and develop it into a practical digitised WBL administrative system that would not
only satisfy the needs of the SWASW partnership, but would also be of value to other
institutions addressing similar issues. In that context it directly contributed to the objectives
of the e-Learning Embedding Benefits programme.
3. Aims and Objectives
The broad aim of the PotsPan project was to implement the digitised online WBL
administrative system, created by the Swani project. The key objectives were to:
Refine, update and test the WBL document management system to ensure it met the
efficiency, usability and data consistency design requirements;
Test and evaluate the system with cohorts of work based learners in the SAWSW
partnership;
Explore the use of digital pens and electronic signatures as part of the WBL
programme audit arrangements;
Contribute to the European debate about digitised document management and the
use of electronic signatures for audit purposes.
4. Methodology
The PotsPan project methodology followed a straightforward testing and evaluation process
that began with a period of system refinement at Coleg Sir Gâr where the document
management system was hosted. The web designer and the project manager worked
together to test each of the system functions and the usability of each of the forms, including
how effectively they shared common data fields.
Workshops were held at both Coleg Sir Gâr and Pembrokeshire College with the WBL
teams that would be testing the system in the field. Each team selected an appropriate
cohort of work based learners and completed a testing and evaluation exercise that involved
completing all the digitised versions of the administrative documents in parallel with
completing the existing paper based versions. Each exercise concluded with a comparison
of the two approached with the aim of judging the relative advantages in terms of efficiency
and effectiveness. These comparisons would be presented in the form of case studies.
An additional exercise was carried out at Swansea Metropolitan where the use of electronic
signatures was explored. It was felt that this solution for document verification would be
eventually accepted by the EU for audit purposes, but that was not the case currently. The
exercise at Swansea Met, being unconstrained by EU rules, was designed to take that
agenda forward by testing and evaluating current electronic signature technologies and their
appropriateness for educational administration, particularly in the distance learning context.
1
http://www.jisc.ac.uk/whatwedo/programmes/elearning/swaniltig/swani.aspx

5
5. Implementation
Following the usability testing exercise at Coleg Sir Gâr, the partner cohorts for the WBL
implementation testing and evaluation were chosen and the process begun. The paper
based administration forms were completed as normal and were then transcribed onto the
digital system. The ease with which this was completed was noted, as was the potential
saving in time with the sharing of common data fields.
A perception at the outset was that there were administrative processes, particularly with the
registration of new learners, for example, where multiple forms were completed by hand that
contained the same information in terms of learner name, address, NI number etc. An
anticipated benefit was that the digitised system would only require these data fields to be
entered once, would ensure data consistency and, where editing was needed at a later date,
the edits would automatically update all forms that used that data.
The testing of electronic signature usage at Swansea Metropolitan was focussed on the
development of a distance learning version of a WBL programme delivered to the armed
forces in Germany. Of particular interest was the use of such signatures to verify the
authenticity of written assessment materials submitted online.
The outcomes of the testing were recorded using a common case study template for ease of
comparison. The outcomes were also shared on the project blog and wiki and were
presented for wider dissemination at ALT-C and at the JISC Online Conference 2012.
6. Outputs and Results
The Outputs and Results for the JISC PotsPan project included an updated and tested WBL
Document Management System, a number of Case Studies that detail the evaluation and
testing of that system, and a project Blog that provides a narrative account of how the project
evolved. The final output was this report that provides an overall summary of activities,
achievements and deliverables.
The primary deliverable was the JOOMLA based document management system that was
originally developed during the Swani project. As a result of the PotsPan testing and
evaluation activities the presentation of the documents was refined and the efficiency and
consistency of data sharing was improved. The system can be viewed at
http://sawsw.colegsirgar.ac.uk/.
Examples of the documents and the way they are presented are shown on the project blog
at http://potspan.blogspot.co.uk/ and are available for download from the project Wiki at
http://potspan.pbworks.com.
There were three structured testing and evaluation exercises planned as part of the PotsPan
project. Two of them were designed to test the document management system, and the use
of the digital pens for audit purposes, with selected WBL cohorts at two of the partner
institutions. The third exercise explored the use of electronically signed documents, rather
than digital pens, in the administration of a work based distance learning programme at a
third partner institution.
The outcomes are presented in the form of case studies. Single page summaries are
included below and the full case studies are included as appendices and can be downloaded
from the project Wiki.

6
Case Study 1: Coleg Sir Gâr
The evaluation at Coleg Sir Gâr involved a group of modern apprentices at the TATA steel
plant in Llanelli and the digitised administrative system was carried out in parallel with the
existing paper based system and the outcomes compared.
Summary
This pilot exercise took place at Tata Steel, Port Talbot and involved Functional Trainees
who are candidates on the NVQ Level 3 and VRQ Level 3 Business Improvement
Techniques Courses. The pilot period started in November 2012.
The cohort chosen for this pilot exercise was made up of 2 groups of functional trainees
studying for an apprenticeship framework made up of an NVQ level 3 and a Technical
Certificate (VRQ Level 3) in Business Improvement Techniques as well as the requirements
for 5 specific units of Key Skills, which may have been achieved previously on another
course of study or would have to be integrated into the current qualification structure.
The students attended tutorials and were supported by a Tutor/Assessor as well as having
review sessions with an Advisor every 56 to 60 days. As well as this, a Mentor was identified
for each candidate at their place of work to support them in attaining the skills required in
their vocational area.
This was carried out using 2 groups of Functional Trainees who had started their courses 4
months apart. Group 2 had started their course of studies in October 2012 while Group 3
started in January 2013.
Therefore a wider variety of forms have been completed for Group 2 compared to Group 3.
The success of the ‘pilot’ was dictated by the availability of the candidates to attend their
review sessions which are planned for every 56 days, with a maximum 60 days between
reviews.
The most important forms for the ongoing process are the Review Forms and the Forward
Learning Plan which are completed every review session.
Due to the logistical problems of using the Digital System on a regular basis during the
review sessions, because of lack of access to the internet system in the workplace, the
existing paper based system was used during these sessions.
The information was then transferred to the digitised administration process at the college or
at home, and the forms then taken back for the candidates to check the accuracy of the
information, and on agreement, to sign the form with the digital pen.
Conclusions
The Digital System was a more organised and efficient system overall, than the existing
paper based system. The Digital Pen worked well. However to ensure that maximum benefit
is achieved from the system it is vital that the system can be accessed directly when carrying
out inductions and reviews on site in companies.
The number of forms on the Digital system could be streamlined for simplicity because the
system lends itself for dividing a form into different sections within a form.
Full Report
The full case study report can be viewed in Appendix A.

7
Case Study 2: Swansea Metropolitan
The evaluation at Swansea Metropolitan was different to the other partners in that it explored
the use of electronic signatures for audit documents rather than digital pens.
Summary
The contribution to be made by Swansea Metropolitan University as a partner in the JISC
PotsPan project was to pilot the use of digital signatures with management and
administrative documents in the context of its merger with University of Wales Trinity Saint
David during the project period. The intention was to explore the benefits of electronic
signatures on documents that needed to be shared on multiple sites, particularly with regard
to the delivery of work-based learning.
The importance of this pilot exercise was that it was not constrained by the EU requirement
for hand written signatures that led to the use of digital pens for the other pilot exercises. It
was therefore an opportunity to explore the use and effectiveness of secure electronic
signatures on documents and how they could be used to authenticate digitally
communicated administrative paperwork.
The basic features of a secure electronic signature system are that it should be able to
confirm ownership of the document, that the signatory was authorised to sign the document
and that the document has not been changed since the signature was applied. Typically the
system will also identify the computer used to create the document and will verify the date
and time the signature was added.
A further element of security in the systems accepted by international finance and commerce
is the use of a third party Certification Authority (CA) that generates encrypted public and
private key certificates. The private key certificate is held on the owners’ computer and
ensures that the encrypted data includes verification information that is recognised by the
receiving computer(s) with the public key. If any aspect of the document and signature
security is not verified, then a ‘not valid’ alert will be shown.
Both commercial and open source document creation applications are available that include
the ability to add secure electronic signatures to documents and provide CA services. An
objective in the PotsPan project was to identify the most cost effective solution for the
institutions and this case study shows how open source solutions are both available and
effective.
The pilot exercise created and tested a series of documents with secure electronic
signatures. These documents were sent electronically as attachments and uploaded to
institutional websites and were shown to retain the encrypted data confirming signature and
document validity.
Conclusions
It was concluded that the work confirmed that electronic signatures that authenticated
digitally transmitted administrative documents were achievable in a cost effective way and
that met commercially accepted levels of verification and security. It is believed that
electronic signatures will eventually become acceptable for EU audit purposes and allow a
fully digitised online administrative system to be implemented.
Full Report
The full case study report can be viewed in Appendix B.

8
Case Study 3: Pembrokeshire College
The evaluation at Pembrokeshire College was also planned to involve a group of work based
learners registered through SAWSW and supported by staff at the college. However, there
were technical and operational problems that prevented the completion of the exercise.
Nonetheless, useful lessons were learned.
Summary
The exercise at Pembrokeshire College began with two workshops, one at Coleg Sir Gâr
involving WBL management staff and the other at Pembrokeshire College that included staff
responsible for WBL learner support.
On both occasions the two key benefits of the digitised document management system were
recognised and welcomed. These being the significant saving in data entry time by the
sharing of core data fields across all the administrative forms, and the information made
available to central management about the whole delivery system at any point in time.
It was agreed that a similar exercise to that at Coleg Sir Gâr would be carried out. A cohort
of students would be identified and they, their employers, and the training advisors
responsible for their support would be registered on the system. Pembrokeshire College was
set up on the digitised administrative system as a training provider with the capacity to self-
register advisors, employers and learners.
Digital pens were supplied to the staff at Pembrokeshire College for the signing of forms and
the digital recording of those signatures. It was at this point that technical barriers to
progress began to emerge. For the pens to be used, the software for the creation of the
digital version of the forms, and the printing of the dot pattern for the hard copy versions, had
to be downloaded and installed. This proved to be highly problematic, possibly due to the
security restrictions imposed by the institutional computer network.
This was not a problem experienced by Coleg Sir Gâr, but it does demonstrate how systems
need to be accommodated by different institutional security regimes for them to be usable by
institutions generally.
A workaround solution was still being developed during the time scheduled for the pilot
exercise and, as a result, the pilot outcomes were not available for this project report.
However, it is planned that the pilot will be completed and this report updated outside the
project period.
Conclusions
It was disappointing that the exercise experienced such difficulties in this instance, but it did
emphasise the importance of ensuring compatibility when adopting third party applications
for inclusion in institutional software solutions.
To some extent, the problems identified here reinforced the conviction that the real solution
to the problem was the acceptance of electronic signatures on documents. This would
remove the need for the hardware interface of the digital pen and its associated software.
It is interesting to note that a new innovation project involving the Welsh Government has
recently been approved at Swansea Metropolitan with all three partners using electronic
signatures.
Full Report
The full case study report will be made available when the pilot exercise is completed and
will be added as Appendix C.

9
7. Outcomes
It is anticipated that the outcomes of the PotsPan project will have practical value, both for
the SAWSW WBL partnership, and for other groups looking for similar administrative
efficiencies using technology. However, there were certain compromises that needed to be
taken for the project to achieve its objectives.
The project took a pragmatic approach to the digitisation design, given the non-acceptance
by the EU of electronic signatures for audit purposes. Ideally, it would have preferred a fully
electronic signature system to be accepted, but, given the current rules, it adopted the digital
pen as a compromise solution.
This was not ideal as it still required paper versions of all the forms to be produced in order
for them to be physically signed. It did allow, however, the entire system to be digitised whilst
still satisfying EU requirements. This meant that the advantages of single data entry, data
consistency and the ability for management to access the system online were achieved.
This latter factor was perhaps one of the most important from a benefit point of view. The
paper based system was managed by Training Advisors and the documentation distributed
geographically as a result. This was both a risk and made central management access to
data very difficult.
However, the goal of achieving an acceptable electronic signature solution remains. The
outcomes of the PotsPan project are being disseminated with a view to contributing to and
furthering that debate. Presentations at Alt-C 2012 & the JISC Online Conference have been
part of that agenda.
8. Conclusions and Recommendations
The outcomes of the PotsPan project led the project team to conclude that the work had
confirmed the main benefits of administrative system digitisation:
Digitisation of the administrative system eliminated the need for the repeated entry of
core data and ensured data consistency;
Digitisation made the entire administrative system available to programme managers
at all times and from any location;
The system delivered significant operational efficiencies, mainly from the avoidance
of physically transporting learner files between different geographical locations.
However, the requirement for physical signatures on audit documents compromised the full
realisation of those benefits and the project team remained committed to advancing the
debate and argument for full electronic signature recognition for EU funded programmes.
The overall conclusion was that the PotsPan project had achieved its primary goal of
exploiting the benefits of the original Swani project and demonstrating its value and viability
in the practical administration and management of the work-based learning programme in
South West Wales. Furthermore, it was felt that the benefits were transferable to other
programmes with similar needs and hence contributed positively to the Jisc e-Learning
Programme Embedding Benefits objectives. However, it was recognised that problems can
arise from using software solutions that are incompatible with institutional systems.
The recommendations arising from the project outcome included:
Noting that not only was a digitised administrative system beneficial from an
efficiency point of view, but also from a cost effective viewpoint: the document
management system used by the PotsPan being open source;
Noting also that there were open source electronic signature systems available that
met standard international security and authentication requirements;

10
Recommending that the current EU debate about the acceptability of electronic
signatures be concluded, not only for educational audit purposes, but for advancing
the wider free market goals of the union.
References and Appendices
All the project documents and additional materials are referred to as links in the text of this
report and/or are available on the project Wiki. For ease of access, the following links lead
directly to the key project documents.
1. PotsPan project Wiki: http://potspan.pbworks.com
2. PotsPan project Blog: http://potspan.blogspot.co.uk
3. Project Management:
http://potspan.pbworks.com/w/page/53357624/Project%20Management
4. The Digitised Administration System:
http://potspan.pbworks.com/w/page/53357697/Digitised%20WBL%20Administration
%20System
5. The Case Studies:
http://potspan.pbworks.com/w/page/53357795/Project%20Deliverables
Appendix A: Coleg Sir Gâr Case Study
Partner Institution: Coleg Sir Gâr
Date: April 2013
Case Study Title: Pilot of the SAWSW Digitised WBL System at TATA
1. Summary
This pilot exercise took place at Tata Steel, Port Talbot and involved Functional Trainees
who are candidates on the NVQ Level 3 and VRQ Level 3 Business Improvement
Techniques Courses. The pilot period started in November 2012
2. The Work-Based Learning Context
The cohort chosen for this pilot exercise was made up of 2 groups of functional trainees
studying for an apprenticeship framework made up of an NVQ level 3 and a Technical
Certificate (VRQ Level 3) in Business Improvement Techniques as well as the requirements
for 5 specific units of Key Skills, which may have been achieved previously on another
course of study or would have to be integrated into the current qualification structure.
The students attended tutorials and were supported by a Tutor/Assessor as well as having
review sessions with an Advisor every 56 to 60 days. As well as this, a Mentor was identified
for each candidate at their place of work to support them in attaining the skills required in
their vocational area.
3. The Pilot Exercise
This was carried out using 2 groups of Functional Trainees who had started their courses 4
months apart. Group 2 had started their course of studies in October 2012 while Group 3
started in January 2013.
Therefore a wider variety of forms have been completed for Group 2 compared to Group 3.
The success of the ‘pilot’ was dictated by the availability of the candidates to attend their
review sessions which are planned for every 56 days, with a maximum 60 days between
reviews.
The most important forms for the ongoing process are the Review Forms and the Forward
Learning Plan which are completed every review session.

11
Due to the logistical problems of using the Digital System on a regular basis during the
review sessions, because of lack of access to the internet system in the workplace, the
existing paper based system was used during these sessions.
The information was then transferred to the digitised administration process at the college or
at home, and the forms then taken back for the candidates to check the accuracy of the
information, and on agreement, to sign the form with the digital pen.
4. The Pilot Outcomes
4.1 The Digital WBL System
The forms that were relevant to the candidates at this particular stage of their course and
apprenticeship were completed on a paper based system and transferred to the Digital
System.
The initial forms were Information sheets with general data to do with employer, college and
courses, learner data and personal information.
The forms that had to be filled in on a regular basis were the ‘Record of Review Form’ and
the ‘Forward Learning Plan’. The Digital system is a more organised and efficient system
than the paper based equivalent but could be further improved in terms of the forms that
have to be filled in on a repetitive basis.
The units being studied on the NVQ 3 and VRQ 3 which are listed on the Individual Learning
Plan could be duplicated on the Record of Review and Forward Learning Plan so that these
don’t have to be input repeatedly on these forms every review. This would enable the
Advisor to update information for the relevant units without having to input the unit title at
every review
4.2 The Digital Pen System
The ‘fastdox’ system was registered so that the digital pen could be used on the relevant
forms that required signatures. The special forms were printed using the ‘fastdox’ virtual
printer which adds dot patterns to documents and forms that will then accept signatures.
The Digital Pen was tested on the Record of Review form and was successfully used by the
candidates and the provider. It was tested by 3 candidates (Samantha Williams, James Petty
and Thomas Courtney) during their review process. The candidates did not have any
difficulty using the pen on the specially prepared forms, and the information transferred
successfully to the software after the pen was docked in the cradle.
The pen will eventually be used on other forms such as Individual Learning Plan, Forward
Learning Plan, Learning Activity Form, Induction Questionnaire and Workplace Induction
Checklist.
The Digital System was a more organised and efficient system overall, than the existing
paper based system. The Digital Pen worked well. However to ensure that maximum benefit
is achieved from the system it is vital that the system can be accessed directly when carrying
out inductions and reviews on site in companies.
The number of forms on the Digital system could be streamlined for simplicity because the
system lends itself for dividing a form into different sections within a form. The Learner
Information Form, Learner Eligibility Form, Learning Activity Form and Contract Agreement,
for example, have very little detail on them.
I feel that the system could go further to reduce repetitive input of information particularly on
the Record of Review Form. The units studied on the NVQ 3 and VRQ 3 could be previously
listed on the Record of Review form so that these do not have to be repeated at every
review.

12
Appendix B: Swansea Metropolitan Case Study
Partner Institution: Swansea Metropolitan, University of Wales Trinity St David
Date: January 2013
Case Study Title: Electronic Signatures for Online Distance Learning Document
Authentication
1. Summary
The contribution to be made by Swansea Metropolitan University as a partner in the JISC
PotsPan project was to pilot the use of digital signatures with management and
administrative documents in the context of its merger with University of Wales Trinity Saint
David during the project period. The intention was to explore the benefits of electronic
signatures on documents that needed to be shared on multiple sites, particularly with regard
to the delivery of work-based learning.
The importance of this pilot exercise was that it was not constrained by the EU requirement
for hand written signatures that led to the use of digital pens for the other pilot exercises. It
was therefore an opportunity to explore the use and effectiveness of secure electronic
signatures on documents and how they could be used to authenticate digitally
communicated administrative paperwork.
The basic features of a secure electronic signature system are that it should be able to
confirm ownership of the document, that the signatory was authorised to sign the document
and that the document has not been changed since the signature was applied. Typically the
system will also identify the computer used to create the document and will verify the date
and time the signature was added.
A further element of security in the systems accepted by international finance and commerce
is the use of a third party Certification Authority (CA) that generates encrypted public and
private key certificates. The private key certificate is held on the owners’ computer and
ensures that the encrypted data includes verification information that is recognised by the
receiving computer(s) with the public key. If any aspect of the document and signature
security is not verified, then a ‘not valid’ alert will be shown.
Both commercial and open source document creation applications are available that include
the ability to add secure electronic signatures to documents and provide CA services. An
objective in the PotsPan project was to identify the most cost effective solution for the
institutions and this case study shows how open source solutions are both available and
effective.
The pilot exercise created and tested a series of documents with secure electronic
signatures. These documents were sent electronically as attachments and uploaded to
institutional websites and were shown to retain the encrypted data confirming signature and
document validity.
It was concluded that the work confirmed that electronic signatures that authenticated
digitally transmitted administrative documents were achievable in a cost effective way and
that met commercially accepted levels of verification and security.
2. The Work-Based Learning Context
Swansea Metropolitan based the electronic signature pilot exercise on plans for the online
distance learning delivery of logistics and transport training for British Forces stationed in
Germany and other operational areas. The courses, validated by the Chartered Institute of
Logistics and Transport2
, are currently delivered on-site by peripatetic tutors and are
consequently expensive to deliver. Online delivery would be much more cost effective, but
2
http://courses.independent.co.uk/training-provider/chartered-institute-of-logistics-transport-13079

13
would require the authentication of work submitted online by the work based learners. This
would normally be in the form of assignments uploaded to the Swansea Metropolitan Moodle
site, accompanied by front sheets signed by an accredited local work-based mentor.
The planning for the online distance learning version of the CILT course was carried out in
parallel with another project funded under the Embedding Benefits strand of the JISC e-
Learning Programme. The Dewi project3
, led by Cardiff Metropolitan University, had the
objective of using the JISC WBL Maturity Toolkit4
to assist in the design of WBL programmes
in the partner institutions. At Swansea Metropolitan the exercise centred on the CILT online
delivery proposal5
and hence fitted in well with the PotsPan pilot. A third project at Swansea
Metropolitan, again part of the the JISC Embedding Benefits programme, also contributed to
the pilot exercise. The was the Smudie project6
which was using Enterprise Architecture and
similar modelling techniques to plan improved student information management systems at
the university and thus also aligned well with the pilot exercise objectives.
3. The Pilot Exercise
The pilot exercise was carried out in three phases. The first was an initial research phase
which explored the methods used to securely authenticate digitised documents and the
signatures that verified them in current global commerce and finance. This phase had the
goal of identifying established trusted practice that could be replicated in an educational
authentication and audit context.
Once established practice appropriate for educational use had been identified, the second
phase objective was to design an implementation plan that satisfied the academic quality
assurance requirements as well as optimising system cost effectiveness. This involved
exploring and testing appropriate implementation options and arriving at a solution that met
both conditions.
The final phase of the pilot exercise was to implement, test and evaluate the chosen option
and to draw conclusions about future viability, sustainability and value in supporting future
online distance learning for the institution. It was also hoped that the outcomes would be of
value to others in the sector considering similar issues.
Phase 1: initial research
Secure electronic signatures are accepted for financial and business transactions globally7
and virtually any level of security can be included in document management workflows.
Adobe summarise the capability well in their introduction to the use of electronic signatures8
using their Acrobat PDF authoring application:
Digital signature capabilities allow authors to set up a secure signing environment and create
simple documents and complex forms with one or more fields. Document authors can design
documents with multiple signature fields each with unique behavioural characteristics and
appearances.
A signed field can lock other fields so that signed data can’t be changed, and authors can
force certain signature fields to be a required part of a workflow. Attention to signature field
design and configuration can help make the document “do the right thing” when someone
receives it as well as control what that person can and cannot do with it.
3
http://www.jisc.ac.uk/whatwedo/programmes/elearning/embeddingbenefits2012/dewi.aspx
4
http://wbltoolkit.pbworks.com/w/page/35396849/Home%20page%20-%20WBL%20Maturity%20Toolkit
5
http://swanseametwbl.pbworks.com/w/page/61324658/JISC%20Dewi%20Project%20Information%20and%20Documents
6
http://smudieprojectblog.blogspot.co.uk/
7
Toole, A. M. (2012) Making Your Mark – Digital Signatures. JISC Innovating e-Learning Online Conference. November 2012.
8
Adobe Systems Inc. (2011) Digital Signatures Enterprise User Guide. [online]. Available at: http://www.adobe.com/devnet-
docs/acrobatetk/tools/DigSig/Acrobat_DigSig_WorkflowGuide.pdf

14
Similarly, the Open Office suite of open source office applications includes the ability to add
digital signatures (and encrypt entire documents) in a secure way. They provide a number of
application scenarios9
including one that is entirely relevant to this case study:
Scenario: Education: Signing and encrypting documents in the education area is
interesting, because it can replace the paper process of correcting dissertations, etc.
Students would send their
signed dissertations to professors, who would make annotation, sign these annotation and
send the
signed document back to the student.
Product Requirement 1: Sign Complete Open Office Documents;
Product Requirement 2: Encrypt complete documents;
Product Requirement 3: Protect content via password and allow to add annotations
(comments) or
tracked changes only;
Product Requirement 4: Sign tracked changes or annotations.
Both Adobe and Open Office enable degrees of security to be applied to their documents
and electronic signature systems that meet the authentication needs of the project. The
difference between them is that Adobe is a commercial product and Open Office is an open
source software application and free to use.
Secure and trusted electronic signature systems typically involve the use of third party
Certification Authority (CA) providers who ensure the validity of the digital certificates they
issue for confirming author identity, ensuring document integrity and providing encryption
keys for secure document distribution. The CA providers would carry out a series of
verification tests to prove identity, including those typically used by banks in providing
financial services. In the most secure systems, the CAs would themselves be legally liable
for any fraud carried out through the use of their digital certificates. Again, both commercial
and free to use service provision is available.
Central to both the Adobe and Open Office ability to add electronic signatures to their
documents is the fact that they are in PDF format. PDF documents not only contain the
document content, they also include any meta-data relating to that content, including
electronic signature data, added when the document was created and saved. When opened
in a PDF document reader, the meta-data will be read and, where it relates to a digital
signature, will be able to verify the status of the document, including the validity of the CA
digital certificate.
Phase 2: the implementation plan
PDF file format is commonly used for the transmission of digital documents as it is an
independent format able to be opened by any PDF reader. It is commonly used in education
as a trusted format and would typically be requested for any document upload or email
attachment. It is entirely appropriate for the purpose of assignment submission by online
distance work-based learners.
Online submission would be through the Swansea Metropolitan Moodle site. Each
submission would be a PDF file containing the assessment document prepared by the
learner, together with a top-sheet containing the electronic signatures of the learner and the
WBL mentor responsible for confirming authenticity. The validity of electronic signatures
would be assured by registering all the users with the CA provider.
Both Adobe Acrobat and Open Office Writer were tested as PDF authoring applications and
were both found to be perfectly adequate. Open Office Writer10
was chosen as the preferred
9
Loehmann, F. (2004) Electronic Signatures and Encryption GUI. [online]. Available at:
http://bcn.boulder.co.us/~neal/i2/OpenOffice_Electronic_Signatures_and_Security.pdf
10
http://www.openoffice.org/product/writer.html

15
option as it was open source and could be freely used by the institution, the WBL employer
and the learners. The CA organisation selected for trialling in the pilot exercise was CAcert11
who provide a free to use digital certificate service.
Phase 3: testing and evaluation
To use Open Office Writer, it needs to be downloaded and installed on each computer to be
used12
. Each user then needs to register with CAcert, the sequence being:
Registration as a user and create an account on the CAcert website13
. This involves
entering a user name and password, along with other unique security data;
When registered and logged in, the process is to select ‘new client certificate’ and
then ‘create certificate’;
A security strength level is selected for the encryption key at this stage. For the
exercise, ‘Microsoft enhanced cryptographic provider v1.0’ was chosen;
When the certificate is created it opens in a new window with an invitation and link to import
into the browser. An email is also sent confirming the creation of the certificate, providing a
link to it in the user account, and also a reminder that the CAcert root certificate also needs
to be imported before certificates can be used. A link to this process is also provided.
The installation process begins with the Root Certificate and this is accessed on the CAcert
Root Certificate website14
. There are links on the page to initiate the Root Certificate
download. Once this has been done, the download is imported in the Internet
Options>Content>Personal>Certificates> of the browser using the ‘Trusted Root Certification
Authorities’ tab. The digital certificate can then be imported by clicking on the link in the
certificate page on the CAcert website. The certificate is now ready for use.
With the digital certificate imported, it can be used to add electronic signatures to Open
Office documents. For the purpose of Pilot Exercise 1 a test document was created in Open
Office Writer and the electronic signature process tested.
Once the document had been created the File>Digital Signatures option was selected and
the newly imported digital certificate option appeared in a new window and was activated by
clicking on the Sign Document button. Once signed and saved, whenever the file is opened
it will have the small icon next to the ‘The signatures in this document are valid’ line in the
image below showing in the document toolbar confirming that it is a valid signed document:
When the document is opened and the icon is double clicked, the documentation verification
will be confirmed and the security information can be viewed:
11
https://wiki.cacert.org/FAQ/AboutUs
12
http://www.openoffice.org/download/
13
https://www.cacert.org/index.php?id=1
14
http://www.cacert.org/index.php?id=3

16
If, at any time, the signed document is edited or changed in any way, the electronic signature
will be invalidated and the icon in the document toolbar will disappear. The edited document
can, of course, be re-signed by an authorised signatory.
The system was tested and evaluated using a series of test documents. The test document
above was sent by email as an attachment and when opened in Open Office Writer, it
included the digital signature that confirmed validity and that the document had not been
altered since the signature had been applied.
When opened, the document was in read-only format which further avoids the possibility of
invalidating the signature. The attached file was saved locally and, when opened as a local
file, still retained the digital signature verification. However, it was now in edit mode and, if
editing did take place, then the digital signature was invalidated.
4. The Pilot Outcomes
Having established the viability of combining the open source document creation and editing
application Open Office with the freely available Certification Authority services provided by
CAcert, the next phase of the pilot exercise was to trial the system with University
assessment documents.
The standard assessment front-sheet used by the Faculty of Applied Design and
Engineering at Swansea Met was chosen. It was imported into Open Office Writer and, after
some appropriate formatting changes suitable for export as an OpenDocument Text file, the
document was digitally signed using the authentication certificate set up during the first pilot
exercise. After confirming the validity of the signature, the file was then sent as an email
attachment and, on receipt, the file was saved and opened using Open Office.

17
The received document can be seen below, along with images demonstrating that the digital
signature was valid and verifiable on receipt.
The outcomes of the pilot exercises can therefore be reported to have met all expectations
and demonstrate that the use of electronic signatures can be reliably applied to remotely
submitted assessment documents. It can be further reported that the outcome is achievable
using open source tools and freely available services, thus making the process highly cost
effective and accessible.
The overall conclusions that can be drawn from the outcomes of this pilot exercise are that
they confirmed that electronic signatures that authenticated digitally transmitted
administrative documents were achievable in a cost effective way and that they met
commercially acceptable levels of verification and security.
In the context of the plans for the delivery of work-based learning by online distance
learning, it demonstrated that the system would facilitate the authentication of remotely
submitted assessment materials. Clearly academic quality approval processes would need
to be satisfied for accreditation, but it is felt that a level of quality assurance equivalent to
that of campus based courses could be demonstrated with this system.
It is recommended that the electronic signature system be further tested and that this should
include exploring alternative applications and services to ensure the most robust and cost
effective combination is adopted. It is suggested that this be included in the preliminary trials
of online distance learning WBL delivery at the university.

18
Appendix C: Pembrokeshire College Case Study
Partner Institution: Coleg Sir Gâr
Date: April 2013
Case Study Title: Pilot of the SAWSW Digitised WBL System
To be added.

Pots pan final report

More Related Content

Viewers also liked

Similar to Pots pan final report

More from Tony Toole

Recently uploaded

Pots pan final report