A Typology of Institutional Practices
for the Recognition of Open
Learning in Europe: Some Findings
from the OpenCred Study
By Gabi Witthaus (ILI, University of Leicester)
and Andreia Inamorato dos Santos (European
Commission, JRC IPTS )
OER15, 14-15 April 2015: Cardiff, Wales
OpenCred Study
• Aim: to support European policy development
• May to November 2014
• University of Leicester and European
Commission (JRC IPTS)
• Investigated institutional strategies to
recognise non-formal, open learning
OpenCred methodology
• Desk research – MOOC portals, journals, e-mails
to colleagues across EU
• Six interviews:
– 2 MOOC teachers
– 2 MOOC learners
– 2 employers/employer bodies
• Analysis
– Identification of key factors that influence recognition
– Development of diamond model
– Analysis of cases using diamond model
First key finding:
There are degrees of formality of recognition
Formality of recognition
Level Descriptors
0 No recognition
1 Completion certificate/badge
2 Certificate/badge with online identity verification (e.g.
Signature Track, Accredible)
3 - Exemption from entrance exam
- Certificate conferring up to 4 ECTS credits
4 - Certificate conferring a minimum of 5 ECTS credits
- Exemption from a course at issuing institution
- ‘Gold standard’ certificate (information, ID verification,
supervision)
- Continuing professional development (CPD) credits
‘Gold standard’ certificates
Certificate from an accredited institution which:
a) ‘formally and clearly states on whose authority it was
issued, provides information on the content, level and
study load, states that the holder has achieved the desired
learning objectives, provides information on the testing
methods employed and lists the credits obtained,
according to a standard international system or in some
other acceptable format
b) is demonstrably and clearly based on authentication [i.e.
student’s identity is verified] and
c) states that the examinations have been administered
under supervision and specifies the nature of this
supervision.’ (NVAO 2014, p.9)
Second key finding
Factors that have the greatest impact on
formality of recognition are:
1. Robustness of assessment
2. Affordability of assessment for learners
3. Learners’ eligibility for assessment
Robustness of assessment
Level Descriptors
0 No assessment
1 - Record of completion of activities
- Self-assessment
- Automated checking, e.g. MCQs (No ID verification)
- Peer assessment (No ID verification)
2 Online exam with ID verification but no real-time
supervision (e.g. Signature Track, Accredible)
3 - Submission of coursework where student is personally
known to examiner (f2f or online)
- Online exam with ID verification and real-time
proctoring (e.g. ProctorU, Proctor2Me, Remote Proctor)
4 - On-site exam (including ‘challenge exams’)
- RPL conducted by recognised experts
Affordability for learners
(cost of assessment/ certificate)
Level Descriptors
0 150 EUR or more
1 81 to 149 EUR
2 20 to 80 EUR
3 1 to 19 EUR
4 No cost to learners
Eligibility for assessment/recognition
Level Descriptors
0 No assessment
1 Only members of a specified group/ profession are
eligible for completion certificates or badges (CPD)
2 Credit-bearing exams only for registered students
3 Exam available to all, but only students enrolled on
programme are eligible for academic credit
4 Everyone is eligible for assessment and recognition
OpenCred Diamond
0
1
2
3
4
Formality of
recognition
Affordability for
learner
Robustness of
assessment
Eligibility for
assessment/recogniti
on
Third key finding
Several different diamond-shaped models
emerged, representing different types of open
courses
Examples of recognition models
1. Typical MOOC with little or no recognition
2. Freemium MOOC: learner pays for
assessment
3. MOOC with recognition for enrolled students
1. Typical MOOC (little/no recognition)
E.g. CARNET (Croatia) MOOC on Developing Courses in Moodle
0
1
2
3
4
Formality of
recognition
Affordability for
learner
Robustness of
assessment
Eligibility for
assessment/recogniti
on
2. Freemium-model MOOC
E.g. University of Osnabrueck MOOC on Data Structures & Algorithms
0
1
2
3
4
Formality of
recognition
Affordability for
learner
Robustness of
assessment
Eligibility for
assessment/recogniti
on
3. MOOC with recognition for enrolled
students
E.g. University of Nicosia MOOC on Digital Currencies
0
1
2
3
4
Formality of
recognition
Affordability for
learner
Robustness of
assessment
Eligibility for
assessment/recogniti
on
Conclusions
• Robust assessment is central to recognition
– Institutions either pass on the cost to learners or
restrict eligibility.
• Recognition only partial – no whole degrees yet
• Online education and assessment still seen by
many as less rigorous
• On-site exams with identity validation and real-
time supervision are seen as being most robust
form of assessment
• ECTS credits are not yet a widely accepted
currency for recognition of open learning
Recommendations to HEIs
• Give clear info to learners on open course
portals about assessment and recognition
• Give comprehensive info about assessment on
certificates/ badges
• OpenCred framework – a tool for analysing
your institution’s recognition model
• Collaboration between institutions (e.g.
VMPass, OERu) can help learners reap
benefits of open learning
Key OpenCred References
Andrade, A., Ehlers, U., Caine, A., Carneiro, R., Conole, G. & Kairamo, A. (2011) Beyond OER: Shifting Focus to Open Educational
Practices, OPAL Report 2011. Link
Camilleri, A.F. & Tannhäuser, A.C. (2013) ‘Chapter 4: Assessment and Recognition of Open Learning’, in L. Squires and A. Meiszner
(eds) Openness and Education. Bingley: Emerald Group Publishing Limited, pp.85-118.
Department for Business, Innovation and Skills (2013) Maturing of the MOOC; BIS Research Paper 130, September 2013. Link
European Commission/EACEA/Eurydice (2014). Modernisation of Higher Education in Europe: Access, Retention and Employability
2014. Eurydice Report. Luxembourg: Publications Office of the European Union
Eurotech Universities (2014) EuroTech Universities session animates debate on MOOCs and future of education at ESOF2014, 7
July 2014. Link
Gaebel, M. (2014) MOOCs: Massive Open Online Course, EUA occasional papers. Link
Khalil, H. & Ebner, M., 2014. MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review. In World
Conference on Educational Multimedia, Hypermedia and Telecommunications. Tampere: Association for the Advancement of
Computing in Education (AACE), Chesapeake, VA, pp. 1305–1313. Link
NVAO (2014) MOOCs and Online HE: A Survey, The Hague: Accreditation Organisation of the Netherlands and Flanders Link
Open Education Special Interest Group (2014) Open Education Trend Report, SURF, Netherlands. Link
Verstelle, M., Schreuder, M. & Jelgerhuis, H., 2014. Recognition of MOOCs in the Education Sector. 2014 Open Education Trend
Report, (March), pp.24–25. Link
For further info
Witthaus, G., Childs, M., Nkuyubwatsi, B.,
Conole, G., Inamorato Dos Santos, A. & Punie, Y.,
2015. An Assessment-Recognition Matrix for
Analysing Institutional Practices in the
Recognition of Open Learning. eLearning Papers,
January (40), pp.32–42. Link
Disclaimer: The views expressed here are purely those of the authors and may not in
any circumstances be regarded as stating an official position of the European
Commission.
Thank you!
And finally…
© G. Witthaus on Flickr CC-BY

A Typology of Institutional Practices for the Recognition of Open Learning in Europe: Some Findings from the OpenCred Study

  • 1.
    A Typology ofInstitutional Practices for the Recognition of Open Learning in Europe: Some Findings from the OpenCred Study By Gabi Witthaus (ILI, University of Leicester) and Andreia Inamorato dos Santos (European Commission, JRC IPTS ) OER15, 14-15 April 2015: Cardiff, Wales
  • 2.
    OpenCred Study • Aim:to support European policy development • May to November 2014 • University of Leicester and European Commission (JRC IPTS) • Investigated institutional strategies to recognise non-formal, open learning
  • 3.
    OpenCred methodology • Deskresearch – MOOC portals, journals, e-mails to colleagues across EU • Six interviews: – 2 MOOC teachers – 2 MOOC learners – 2 employers/employer bodies • Analysis – Identification of key factors that influence recognition – Development of diamond model – Analysis of cases using diamond model
  • 4.
    First key finding: Thereare degrees of formality of recognition
  • 5.
    Formality of recognition LevelDescriptors 0 No recognition 1 Completion certificate/badge 2 Certificate/badge with online identity verification (e.g. Signature Track, Accredible) 3 - Exemption from entrance exam - Certificate conferring up to 4 ECTS credits 4 - Certificate conferring a minimum of 5 ECTS credits - Exemption from a course at issuing institution - ‘Gold standard’ certificate (information, ID verification, supervision) - Continuing professional development (CPD) credits
  • 6.
    ‘Gold standard’ certificates Certificatefrom an accredited institution which: a) ‘formally and clearly states on whose authority it was issued, provides information on the content, level and study load, states that the holder has achieved the desired learning objectives, provides information on the testing methods employed and lists the credits obtained, according to a standard international system or in some other acceptable format b) is demonstrably and clearly based on authentication [i.e. student’s identity is verified] and c) states that the examinations have been administered under supervision and specifies the nature of this supervision.’ (NVAO 2014, p.9)
  • 7.
    Second key finding Factorsthat have the greatest impact on formality of recognition are: 1. Robustness of assessment 2. Affordability of assessment for learners 3. Learners’ eligibility for assessment
  • 8.
    Robustness of assessment LevelDescriptors 0 No assessment 1 - Record of completion of activities - Self-assessment - Automated checking, e.g. MCQs (No ID verification) - Peer assessment (No ID verification) 2 Online exam with ID verification but no real-time supervision (e.g. Signature Track, Accredible) 3 - Submission of coursework where student is personally known to examiner (f2f or online) - Online exam with ID verification and real-time proctoring (e.g. ProctorU, Proctor2Me, Remote Proctor) 4 - On-site exam (including ‘challenge exams’) - RPL conducted by recognised experts
  • 9.
    Affordability for learners (costof assessment/ certificate) Level Descriptors 0 150 EUR or more 1 81 to 149 EUR 2 20 to 80 EUR 3 1 to 19 EUR 4 No cost to learners
  • 10.
    Eligibility for assessment/recognition LevelDescriptors 0 No assessment 1 Only members of a specified group/ profession are eligible for completion certificates or badges (CPD) 2 Credit-bearing exams only for registered students 3 Exam available to all, but only students enrolled on programme are eligible for academic credit 4 Everyone is eligible for assessment and recognition
  • 11.
    OpenCred Diamond 0 1 2 3 4 Formality of recognition Affordabilityfor learner Robustness of assessment Eligibility for assessment/recogniti on
  • 12.
    Third key finding Severaldifferent diamond-shaped models emerged, representing different types of open courses
  • 13.
    Examples of recognitionmodels 1. Typical MOOC with little or no recognition 2. Freemium MOOC: learner pays for assessment 3. MOOC with recognition for enrolled students
  • 14.
    1. Typical MOOC(little/no recognition) E.g. CARNET (Croatia) MOOC on Developing Courses in Moodle 0 1 2 3 4 Formality of recognition Affordability for learner Robustness of assessment Eligibility for assessment/recogniti on
  • 15.
    2. Freemium-model MOOC E.g.University of Osnabrueck MOOC on Data Structures & Algorithms 0 1 2 3 4 Formality of recognition Affordability for learner Robustness of assessment Eligibility for assessment/recogniti on
  • 16.
    3. MOOC withrecognition for enrolled students E.g. University of Nicosia MOOC on Digital Currencies 0 1 2 3 4 Formality of recognition Affordability for learner Robustness of assessment Eligibility for assessment/recogniti on
  • 17.
    Conclusions • Robust assessmentis central to recognition – Institutions either pass on the cost to learners or restrict eligibility. • Recognition only partial – no whole degrees yet • Online education and assessment still seen by many as less rigorous • On-site exams with identity validation and real- time supervision are seen as being most robust form of assessment • ECTS credits are not yet a widely accepted currency for recognition of open learning
  • 18.
    Recommendations to HEIs •Give clear info to learners on open course portals about assessment and recognition • Give comprehensive info about assessment on certificates/ badges • OpenCred framework – a tool for analysing your institution’s recognition model • Collaboration between institutions (e.g. VMPass, OERu) can help learners reap benefits of open learning
  • 19.
    Key OpenCred References Andrade,A., Ehlers, U., Caine, A., Carneiro, R., Conole, G. & Kairamo, A. (2011) Beyond OER: Shifting Focus to Open Educational Practices, OPAL Report 2011. Link Camilleri, A.F. & Tannhäuser, A.C. (2013) ‘Chapter 4: Assessment and Recognition of Open Learning’, in L. Squires and A. Meiszner (eds) Openness and Education. Bingley: Emerald Group Publishing Limited, pp.85-118. Department for Business, Innovation and Skills (2013) Maturing of the MOOC; BIS Research Paper 130, September 2013. Link European Commission/EACEA/Eurydice (2014). Modernisation of Higher Education in Europe: Access, Retention and Employability 2014. Eurydice Report. Luxembourg: Publications Office of the European Union Eurotech Universities (2014) EuroTech Universities session animates debate on MOOCs and future of education at ESOF2014, 7 July 2014. Link Gaebel, M. (2014) MOOCs: Massive Open Online Course, EUA occasional papers. Link Khalil, H. & Ebner, M., 2014. MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review. In World Conference on Educational Multimedia, Hypermedia and Telecommunications. Tampere: Association for the Advancement of Computing in Education (AACE), Chesapeake, VA, pp. 1305–1313. Link NVAO (2014) MOOCs and Online HE: A Survey, The Hague: Accreditation Organisation of the Netherlands and Flanders Link Open Education Special Interest Group (2014) Open Education Trend Report, SURF, Netherlands. Link Verstelle, M., Schreuder, M. & Jelgerhuis, H., 2014. Recognition of MOOCs in the Education Sector. 2014 Open Education Trend Report, (March), pp.24–25. Link
  • 20.
    For further info Witthaus,G., Childs, M., Nkuyubwatsi, B., Conole, G., Inamorato Dos Santos, A. & Punie, Y., 2015. An Assessment-Recognition Matrix for Analysing Institutional Practices in the Recognition of Open Learning. eLearning Papers, January (40), pp.32–42. Link
  • 21.
    Disclaimer: The viewsexpressed here are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission. Thank you!
  • 22.
    And finally… © G.Witthaus on Flickr CC-BY

Editor's Notes

  • #2 This paper reports on a study carried out from May to November 2014, as part of an investigation into institutional strategies used by European universities for opening up education, including the challenges and opportunities in the recognition of open learning achievements. The overall aim of the study was to support European policy development. The research team investigated institutional practices, attitudes and rationales for the types of recognition awarded for open learning, the factors that influenced decisions in this regard, and the contexts in which non-formal, open learning was recognised. Desk research was conducted to obtain an overview of which institutions and collaborative groupings of institutions throughout Europe are recognising open learning, and what mechanisms they are using for such recognition. This data was complemented by six in-depth interviews – two with MOOC teachers, two with MOOC learners, and two with employers/employer bodies that were beginning to recognise open learning for continuing professional development purposes. The following key themes emerged from the data gathered: Recognition is not a monolith, but rather, it manifests at several levels of formality, from badges/completion certificates up to verified certificates conferring ECTS credits.The aspect of open learning that has the greatest impact on formality of recognition is the nature of the assessment implemented. Critical factors in determining robustness of assessment are identity verification, real-time supervision during examinations (whether conducted online or face-to-face), and the inclusion of comprehensive information about both the course content and the assessment procedures on the certificate.Two other aspects of open learning have a significant impact on recognition – affordability of the assessment for learners, and learners’ eligibility for assessment and recognition. It was found that the MOOC providers that offer full recognition tend to either pass on the cost of the examination to the learners, or to only offer the examination and recognition to students enrolled on a programme at that institution. Based on these findings, a typology of recognition types was generated, in which open learning initiatives are represented in the form of a diamond-shaped radar graph, with formality of recognition, robustness of assessment, affordability of assessment for learners and eligibility for assessment at each of the four points. A number of MOOCs and open learning initiatives were analysed using this tool, and the resultant models fell into six clusters of similar-shaped graphs. These clusters form the core of a new typology for recognition of open learning. It is anticipated that this typology will help academics and senior management in higher education institutions and employer bodies analyse their existing practices regarding recognition of open learning, comparing and contrasting them with those of other institutions, and to develop strategies for enhancing their recognition practices in the future.
  • #4 http://is.jrc.ec.europa.eu/pages/EAP/OpenEdu.html