Invited keynote presentation for HAKA, Estonia, 19th May 2023.

1. 19th May 2023
Professor Mark Brown
Dublin City University
Photo by Hulki Okan Tabak on Unsplash

2. Quality Assurance of Digital Teaching and Learning

3. Quality Assurance of Digital Teaching and Learning

4. Quality Assurance of Digital Teaching and Learning

5. Insights from OECD study
Insights on QA of micro-credentials
Insights from Irish National QA Guidelines
Outline

6. Underlying assumptions

7. Quality concerns are not new
Delivery mode alone is not a key factor
Definition wars are largely unproductive
Underlying assumptions

8. “Importantly, the learner’s
location is a key determinant in
naming the mode”
(Johnson, Seaman & Poulin, 2022, p. 106)
https://olj.onlinelearningconsortium.org/index.php/olj/article/view/3565
On-campus
Off-campus
#3 Definition wars are largely unproductive

9. QA / QE
Gap
Digital
Education
- Macro-Level
- Meso-Level
- Micro-Level
- Nano-Level
 Multifaced and
multifunctional
 Highly contextualised
 Institutions are central
 Owned & distributed
across institutions
 Conversational,
shares experiences
and involves feedback
loops
 Dynamic as part of a
living and thriving
quality culture
Framing principles

10. 1. Insights from OECD study
2022
https://www.oecd-ilibrary.org/education/digital-higher-education_f622f257-en

11. External quality assurance
Institutional quality assurance
Institutional supports to enhance quality
1. Insights from OECD study

12. 1
In 23 OECD jurisdictions, we found no national framework, standards or guidelines for digital higher education,
and no evidence of a decision taken on how to approach the quality assurance of digital higher education.
No approach for the quality assurance of digital higher education
1. Insights from OECD study - External

13. 1
In 23 OECD jurisdictions, we found no national framework, standards or guidelines for digital higher education,
and no evidence of a decision taken on how to approach the quality assurance of digital higher education.
2
In 8 OECD jurisdictions, we found common standards for digital and traditional study modes, with evidence of a
decision to apply the standards for the quality assurance of on-campus instruction to digital higher education.
No approach for the quality assurance of digital higher education
Common standards and guidance for the quality assurance of digital higher education
1. Insights from OECD study - External

14. https://www.teqsa.gov.au/online-learning-good-practice
Additional Guidance
1. Insights from OECD study - External

15. https://www.qaa.ac.uk/docs/qaa/guidance/questions-to-inform-a-toolkit-for-enhancing-quality-in-a-digital-environment.pdf?sfvrsn=4fabcf81_6
2020
1. Insights from OECD study - External

16. 1
In 23 OECD jurisdictions, we found no national framework, standards or guidelines for digital higher education,
and no evidence of a decision taken on how to approach the quality assurance of digital higher education.
2
In 8 OECD jurisdictions, we found common standards for digital and traditional study modes, with evidence of a
decision to apply the standards for the quality assurance of on-campus instruction to digital higher education.
3
In 12 OECD jurisdictions, we identified specific standards or guidelines for digital higher education. These either
cover all types of digital education or a specific type (or types) of digital education (e.g., hybrid education).
No approach for the quality assurance of digital higher education
Common standards and guidance for the quality assurance of digital higher education
Specific standards for the quality assurance of digital higher education
1. Insights from OECD study - External

17. 2021
Specific
https://mfhea.mt/wp-content/uploads/2021/10/Guidelines-for-FHEI-V1.pdf
1. Insights from OECD study - External

18. 2018
https://www.qqi.ie/sites/default/files/media/file-uploads/Statutory%20QA%20Guidelines%20for%20Blended%20Learning%20Programmes.pdf
1. Insights from OECD study - External

19. 1. Insights from OECD study - Institutional

20. Most
Common
Least
Common
Aggregated summary…
1. Insights from OECD study - Institutional

21. • An abundance of QA frameworks
• Many common dimensions shared across
the different QA frameworks
• Absence of output measures of QA
• Emergence of new QA considerations
• More research needed on how institutions
implement QA/QE frameworks
Findings…
https://www.tandfonline.com/doi/abs/10.1080/08923647.2018.1417658
1. Insights from OECD study - Institutional

22. 12%
2020 Survey
1. Insights from OECD study - Institutional

23. Culture
2022
https://www.tandfonline.com/doi/full/10.1080/08923647.2021.2005414
1. Insights from OECD study - Institutional

24. What else is missing
from current QA frameworks?
(Brown, 2022)
1. Insights from OECD study – Gaps

25. https://www.oecd-ilibrary.org/education/digital-higher-education_f622f257-en
(How) do the quality
assurance principles apply to
micro-credentials?
2. Insights on QA of micro-credentials
OECD, 2022

26. https://www.oecd-ilibrary.org/education/micro-credentials-for-lifelong-
learning-and-employability_9c4b7b68-en
OECD, 2023
2. Insights on QA of micro-credentials

27. March 2023
88%
2. Insights on QA of micro-credentials

28. 25% -
36% -
33% -
6% -
March 2023
2. Insights on QA of micro-credentials

29. 2. Insights on QA of micro-credentials - External
Ireland
https://microbol.microcredentials.eu/outputs/
2021

30. https://www.enqa.eu/events/online-dissemination-event-external-quality-assurance-of-micro-credentials-27-september-2022/
2. Insights on QA of micro-credentials - External

31. Yes
NA
2022
https://www.enqa.eu/events/online-dissemination-event-external-quality-assurance-of-micro-credentials-27-september-2022/
2. Insights on QA of micro-credentials - External

32. 1
In 25+ countries, we found no national framework, standards or guidelines for micro-credentials. In many
cases, however, there is evidence of a plan or commitment in the future to integrating quality assurance standards
for micro-credentials within existing requirements.
No specific quality assurance standards for micro-credentials
2. Insights on QA of micro-credentials - External

33. 1
In 25+ countries, we found no national framework, standards or guidelines for micro-credentials. In many
cases, however, there is evidence of a plan or commitment in the future to integrating quality assurance standards
for micro-credentials within existing requirements.
2
In 6 counties (Australia, Canada (x3), United Kingdom, Estonia, Spain (x2) and The Netherlands
we found common standards already apply for micro-credentials, and tangible evidence of a decision to extend
the application of existing standards for their quality assurance.
No specific quality assurance standards for micro-credentials
Common standards and quality assurance processes apply for micro-credentials
2. Insights on QA of micro-credentials - External

34. 2. Insights on QA of micro-credentials - External

35. 1
In 25+ countries, we found no national framework, standards or guidelines for micro-credentials. In many
cases, however, there is evidence of a plan or commitment in the future to integrating quality assurance standards
for micro-credentials within existing requirements.
2
In 6 counties (Australia, Canada (x3), United Kingdom, Estonia, Spain (x2) and The Netherlands
we found common standards already apply for micro-credentials, and tangible evidence of a decision to extend
the application of existing standards for their quality assurance.
3
In New Zealand, Malaysia and Ireland, we identified a framework or specific standards or guidelines for
micro-credentials to address particular issues related to their quality assurance.
No specific quality assurance standards for micro-credentials
Common standards and quality assurance processes apply for micro-credentials
Specific standards and quality assurance processes developed for micro-credentials
2. Insights on QA of micro-credentials - External

36. 2. Insights on QA of micro-credentials - External
New Zealand
United Kingdom

37. 2. Insights on QA of micro-credentials - Institutional

38. https://www.microcredseeker.edu.au
343
11
12
20/03/2023
93% Online
2. Insights on QA of micro-credentials - Institutional

39. https://cdn.csu.edu.au/__data/assets/pdf_file/0003/4137924/Micro-Credential-Framework.pdf
2. Insights on QA of micro-credentials - Institutional

40. 2. Insights on QA of micro-credentials - Institutional
https://microcreds.ie/
- 49% Online
On-campus = 0
Hybrid = 0
Online = 123
Blended = 93
15/05/2023

41. https://www.tcd.ie/teaching-learning/academic-affairs/postgraduate-education/microcredentials/§
QC ◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊◊
2. Insights on QA of micro-credentials - Institutional

42. Plan and Adjust
• Institutional leadership
• Organisational structures
• Business models and
resource allocation
• Policies, regulations and
pathways, including global
• IT Systems and platforms
- Flexible enrolment
- Virtual learning
environment (VLE)
- Digital badging/certificate
Implement Monitor
• Internal approval processes
• Appropriate workload
models
• Professional learning and
support for MC
development
• Peer review of learning
design
• Appropriate assessment
and learner feedback
strategies
• Study disclosure and
learner readiness for
success
Quality Assurance Processes
and Supports
Strategy, Quality Culture and
Infrastructure
Feedback and Performance
Reporting
• Learning analytics data on
student engagement
• Retention, progression and
completion data
• Student experience data
• Employer satisfaction data
• Meet professional
accreditation requirements
• Graduate employment data
• Cyclic institutional review of
micro-credential offerings
2. Insights on QA of micro-credentials – Gaps

43. 3. Insights from Irish National QA Guidelines
2018 2023

44. What in a single word is the most
important quality consideration
that you would like to see in the
new QQI guidelines?
3. Insights from Irish National QA Guidelines

45. 3. Insights from Irish National QA Guidelines
Adapted
from
OECD

46. 3. Insights from Irish National QA Guidelines

47. 3. Insights from Irish National QA Guidelines

48. 3. Insights from Irish National QA Guidelines

49. 3. Insights from Irish National QA Guidelines

50. 3. Insights from current Irish national guidelines

51. 3. Insights from current Irish national guidelines

52. 3. Insights from Irish National QA Guidelines

53. 3. Insights from Irish National QA Guidelines
https://www.qqi.ie/what-we-do/engagement-insights-and-knowledge-sharing/current-consultations

54. Specific QA issues for online [MCs]
Organisational context
• Business continuity
• Access to IT infrastructure
• Learners outside of country
Programme context
• Online teaching experience
• Training and professional development
• Synchronous vs asynchronous delivery
Learner context
• Learner readiness
• Access to online resources
• Equivalency of learning support
3. Insights from Irish National QA Guidelines

55. EUROPEAN STANDARDS AND
GUIDELINES
EUROPEAN FRAMEWORKS FOR
DIGITAL HIGHER EDUCATION
COUNTRY SPECIFIC QUALITY
FRAMEWORKS AND GUIDELINES
Final remarks

56. The problem is not
making up the new
dance steps but
deciding which ones to
keep and combine to
help make a great
performance
[experience].
Final remarks…

57. Staring, F., Brown, M., Bacsich, P., & Ifenthaler, D. (2022). Digital higher
education: Emerging quality standards, practices and supports, OECD Education
Working Papers, No. 281, OECD Publishing, Paris,
https://doi.org/10.1787/f622f257-en.
Volungevičienė, A., Brown, M., Greenspon, R., Gaebel, M., & Morrisroe, A.
(2021). Developing a High-Performance Digital Education System: Institutional
Self-Assessment Instruments. European University Association, Brussels.
Ossiannilsson, E., K. Williams, A. Camilleri and M. Brown (2015), Quality models
in online and open education around the globe: State of the art and
recommendations. International Council for Open and Distance Education, Oslo,
Norway. https://files.eric.ed.gov/fulltext/ED557055.pdf
Esfijani, A. (2018), “Measuring quality in online education: A metasynthesis”,
American Journal of Distance Education, 32/1, pp. 57-73, DOI:
10.1080/08923647.2018.1417658
Brown, M. (2022). The quest for quality in digital higher education: A critical
analysis of QA frameworks. Full paper at EDEN Digital Learning Europe Annual
Conference, Tallinn, Estonia, 20th June.
Key references
2022
https://link.springer.com/referenceworkentry/10.1007/978-981-19-0351-9_41-1

58. Thank you
Professor Mark Brown
Director, National Institute for Digital Learning
Dublin City University
Ireland
mark.brown@dcu.ie
Contact details…