‘Getting started with MOOCs’ BLE workshop: 21st May 2019
Case study: MOOCs for professional
development in global eye health
Sally Parsley, Digital Education Manager
Disability & Eye Health Group
Open Education for Eye Health Programme

Case study: Open Education for Eye Health
Aim: To promote an equitable increase in participation in public health eye care training.
Two ‘prongs’:
Collaboratively create & deliver
Open access online courses
(MOOCs) aimed at eye health
teams in low and middle income
countries
Provide support for additional
contextualisation and adoption of
the training by local eye health
educators

1. Analyse
2. Design
3. Implement
4. Realise
Focus on quality to deliver success
5. Evaluate
MOOC Quality Reference
Framework (Stracke et al. 2018)
Product cycle

1. Analyse
2. Design
3. Implement
4. Realise
Focus on quality to deliver success
5. Evaluate
MOOC Quality Reference
Framework (Stracke et al. 2018)
Product cycle

2 important MOOC affordances affecting quality for us:
Openness/Flexibility
Non restrictive registration/free access, use of OER, flexibility of provision have all
contributed to increased access to and participation in the training on and beyond the
platform. Need to design for diverse groups of learners (who to target?)
Autonomous learning
• Self-directed learners have more choice & control – low opportunity cost of drop out
(funnel of participation).
• Consider how to define & measure success for diverse groups of learners
• Design feedback and self-assessment support for diverse groups of learners
1. Consider MOOC’s affordances early

Our experiences:
• Be explicit when identifying your target
learners and stakeholders
• Try to find out:
1. What do learners want (demand)?
2. What do stakeholders need?
• Build on your existing networks: Our MSc
alumni have been fantastic champions for us
• Actively manage communication with your
learners and stakeholders
2. Engage your learners and stakeholders
Eye health
educational
leaders
Eye health
educators &
managers
Programme
funders
LSHTM & DEHG
leadership
Eye health
team
members
People with
eye disease
& families

3. Define & evaluate success
Our experiences
• Providers, learners
& stakeholders have
different views of
success
• Include evaluation
across all processes
• I wish we’d put an
explicit line in the
budget for ‘product
cycle’ evaluation
Metrics and methods we’ve used
• Platform surveys & analytics: Learner coverage, demographics,
participation patterns & satisfaction
• Bespoke post course surveys & interviews:
• Barriers and challenges to participation
• Knowledge/skills expansion & application to practice
• Ad-hoc qualitative feedback and case studies
• Future (?): Bespoke pre-course surveys - learner roles & intentions
Learning evaluation frameworks we’ve used:
• Value creation framework (Wenger et al. 2011), (Kennedy & Laurillard 2019)
• Kirkpatrick (2007)

1-50 101-500 501-1,00051-100 >1,000
Examples of our evaluation metrics
Eye health MOOCs: Joiners by country Mar’16-Jan’19

Learner feedback DED run 1 (Oct/Nov 2018)
55 responses, 37 appreciative
• Thank you for this important course. Diabetes is a challenge in our work places and therefore found this course to be
very helpful
• Thank you a whole lot,I really benefitted a lot from this course ,looking forward to putting what I have learnt to
practice .
• I am a type 1 diabetic and regularly attend the screening clinic and local hospital. This course explained a great deal
about what I should be aware of .
• Much appreciate the extensive content of the course with downloabable materials, videos and links to related online
materials
3 upgrade/certificate related requests
• May I upgrade and get the certificate at a later date? I am quite low on funds presently.
4 suggestions/requests for changes to length or timing of course
• It should be 6 week course instead.
6 suggestions/requests for content changes
• Yes, made this course available in French to reach most people from Sub saharian Africa
• Diabetic Retinopathy is not the only eye complication triggered by diabetes ….

Great courses for
blending into my resident
training programme – high
quality content & practical
examples, global aspect is
another strength
- Uganda
Adapted the GB
content for use in
face to face nurse
training -
Botswana
Educator
feedback
Involve & support
collaborators/educators
more. Co-ownership will
drive local engagement
and leadership buy-in –
(x3)
70% of survey
respondents* had
reused GB1 OER
Internet access remains a
barrier (x3)
No culture
of CPD
here (x2)
Excellent global content - I
contextualise where needed –
blending lets me concentrate on
discussions with my Masters
students - South Africa
Saves
time!
(x2)
Use research
to build
evidence that it
works (x2)
Great course (DED) -
I want formal
accreditation to
promote this as CPD
to my residents
Rwanda
Image: © Marc Wathieu CC BY-NC https://flic.kr/p/5HzQxz

Our experience
• Share (task shift)
workload, encourage
creativity & keep focus
on the learner
• Team approach +
active collaboration
has really
strengthened the
quality of our courses
4. Use a team approach
Our approach
• Small core team managed through a key partnership
between lead academic and learning designer
• Weekly calls and task prioritisation
• Project admin & design tools
• Active inclusion of stakeholders and learner voices:
– Expert steering groups
– Co-creation of content with eye health workers
(learners) and educators
– Feedback from learners within course
– Localisation partnerships
• Moving forward – looking to strengthen inclusion of
learners and stakeholders across our processes

Core
team
34 LSHTM
collaborators
7 PhD & MSc
students
19 faculty
8 learning
tech &
support staff
2 funders
78 external
collaborators
from 25 countries
17 steering
group
members
1 Learning
technologist
64
Contributors
39
educators &
facilitators
Collaborations have been key to success
Our alumni

Examples of content co-creation

3 MOOC learning design methods to explore….
• MOOC Design Mapping Framework
See References for links to the MDMF and Kerr et al.’s (2019) evaluation
See our adapted MDMF template at https://miro.com/app/board/o9J_kycp4vA=/
• ABC LD Workshop – See the introduction by Young & Perovic (2018)
• Learning Designer (Laurillard et al. 2018)
Our approach
1. Team based & learner centred
2. Use learning designs to map learner journeys and
check our use of conversational learning
(Laurillard 2002) & constructive alignment (Biggs
2014)
3. Feed into production tracking documents
5. Foreground pedagogy
Our documentation
A. Course overview documents
(Google)
B. Weekly learning designs (Miro)
C. Script templates & production
sheets (Google)

Examples of our documentation
Course overview docs

Weekly learning design

Production sheet
Schedule

Script templates

6-12 mths
Pre-course
survey
Facilitation
Platform
data
Post course
survey
90% on
platform
Course
opens
Marketing
begins
Create
evaluation
instruments
Summary tasks across MOOC production cycle
3 months
pre launch
2 & 3. Design and create
1 month
pre
launch
4. Realise:
FL
testing
Identify
collaborat
ors
Course
overview
Learning
designs
By week: Script,
storyboard & create
multimedia assets
Write
activities
& discussions
Write
quizzes
Review,
upload
text &
assets &
proof
~3-6 mths
post course
Impact
interviews
Data
analysis
Insights
for action
1. Analysis
Identify:
Need/demand
Context
Target learners
& stakeholders
Team
Budget,
timeline &
resources
5. Evaluation of each stage
Course runs: 4- 6
weeks

In summary
What we’ve learned about delivering quality in MOOCs:
1. Consider your MOOC affordances early
2. Actively engage with your learners & stakeholders
3. Define & then evaluate MOOC success
4. Use a team approach
5. Foreground pedagogy in your design
© 2019 International Centre for Eye Health, London School of Hygiene & Tropical Medicine. Unless otherwise stated this work is licensed under the
Creative Commons Attribution Non-Commercial Share-Alike 4.0 International License

Thank you
We would like to acknowledge all our funders, partners, colleagues and eye health workers who
have contributed to this programme as learners, practitioners, educators and leaders.
We’re always happy to discuss MOOC production, share our tools and methods, and learn from
others!
Contact me:
Sally.Parsley@LSHTM.ac.uk
@sallyeparsley
Find out more about our Open Education for Eye Health programme and download our MOOC
design templates at https://ICEH.lshtm.ac.uk/oer
© 2019 International Centre for Eye Health, London School of Hygiene & Tropical Medicine. Unless otherwise stated this work is licensed under the
Creative Commons Attribution Non-Commercial Share-Alike 4.0 International License

Biggs, J. (2014). Constructive alignment in university teaching. HERDSA Review of Higher Education, 1, 5–22. https://doi.org/10.1046/j.1365-2923.1999.00431.x
Kennedy, E., & Laurillard, D. (2019). A MOOC Value Creation Methodology. London: UCL Institute of Education. https://doi.org/10.4169/college.math.j.46.4.270
Kerr, J., Dale, V. H. M., & Gyurko, F. (2019). Evaluation of a MOOC Design Mapping Framework (MDMF): Experiences of Academics and Learning Technologists. The
Electronic Journal of E-Learning, 17(1), 38–51. Retrieved from https://issuu.com/academic-conferences.org/docs/ejel-volume17-issue1-article659?mode=a_p
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the Four Levels: A Practical Guide for Effective Evaluation of Training Programs. Berrett-Koehler Publishers.
Laurillard, D. (2002) Rethinking university teaching: a conversational framework for the effective use of learning technologies (2nd ed.). London: Routledge Falmer
Laurillard, D., Kennedy, E., Charlton, P., Wild, J., & Dimakopoulos, D. (2018). Using technology to develop teachers as designers of TEL: Evaluating the learning
designer. British Journal of Educational Technology, 49(6), 1044–1058. https://doi.org/10.1111/bjet.12697
Stracke, C. M., Tan, E., Texeira, A., Pinto, M., Vassiliadis, B., Kameas, A., Sgouropoulou, C., & Vidal, G. (2018). Quality Reference Framework (QRF) for the Quality of
Massive Open Online Courses (MOOCs). Retrieved from www.mooc-quality.eu/ QRF
The MOOC Design Mapping Framework (MDMF). Glasgow University College of Social Sciences. Available from:
https://www.gla.ac.uk/colleges/socialsciences/staff/learningandteaching/moocdesign/
Wenger, E., Trayner, B., & De Laat, M. (2011). Promoting and Assessing Value Creation in Communities and Networks: a Conceptual Framework. Heerlen: Ruud de
Moor Centrum, Open Universiteit. Retrieved from http://wenger-trayner.com/resources/publications/evaluation-framework/
Young, C., & Perović, N. (2018). Introduction to the ABC LD workshop. London. Retrieved from
https://docs.google.com/viewerng/viewer?url=http://blogs.ucl.ac.uk/abc-ld/files/2018/05/ABC_LD-Toolkit-Intro.pdf&hl=en
[Accessed 2019.05.18]
References