This document provides an overview of the MOOCKnowledge study being conducted by JRC-IPTS and other partners. The study aims to better understand MOOC learners in Europe through a large-scale survey of learners from different European MOOCs. The survey will collect data at three time points - before, immediately after, and one year after course completion - to understand learner demographics, motivations, experiences, and long-term impacts. Preliminary results from an initial pilot survey of over 1700 learners across 6 MOOCs show distributions of gender, age, education level, and employment status. The findings will help inform evidence-based policy on open education in Europe.

1. OpenEdu
Opening up Higher Education in Europe
A workshop with university leaders from 18 European
countries on the OpenEdu framework for openness
in HE institutions
Andreia Inamorato dos Santos
Yves Punie
Jonatan Castaño Muñoz
@ aisantos @yves998 @jcastanom
#iptsopen
JRC IPTS- Institute for Prospective Technological Studies
The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor
any person acting on behalf of the Commission is responsible for the use which might be made of this publication.
© European Commission
Reuse is authorised provided the source is
acknowledged

2. European Commission,
Joint Research Centre
Institute for Prospective
Technological Studies (IPTS):
Research institute supporting EU
policy-making on
socio-economic, scientific and/or
technological issues

3. IPTS IS Unit work on ICT for
Learning and Skills
Yves Punie, Project Leader
Institute for Prospective Technological Studies, IPTS

4. ICT for Learning and Skills
(2005 - …)
http://essie.eun.org/
• > 80 publications freely downloadable
• Principal clients: DG EAC & DG EMPL
Policy
• 2013 COM on Opening up Education; E&T 2020;
Digital Agenda; New skills and Jobs; EU
Recommendation on Key Competences for
LLL,…
• Juncker priorities: DSM – Jobs, Growth &
Investment
What:
• ICT for modernising and innovating E&T in Europe
• 21st century skills for digital economy and society
Why:
• Evidence for more effective and relevant E&T in Europe

5. DIGITAL Transforming of E&T
DigCompOrg
DigComp
DigCompTeach
OpenEdu Competent
citizens
Competent
Teachers
Competent
Schools
Open Higher
Education
Institutions EntreComp
A coherent approach

6. Innovation and digital
transformation of E&T
Digitally-Competent Educational Organisations
(DigCompOrg)
Mainstreaming ICT-enabled innovation for
learning
(SCALE CCR)

7. Sense of Initiative and Entrepreneurship
framework (EntreComp)
Skills and competences
development
Digital Competence Framework
(DigComp)
5 competence areas 21 competences
1. Information
1.1 Browsing, searching, & filtering information
1.2 Evaluating Information
1.3 Storing and retrieving information
2. Communication
2.1 Interacting through technologies
2.2 Sharing information and content
2.3 Engaging in online citizenship
2.4 Collaborating through digital channels
2.5 Netiquette
2.6 Managing digital identity
3. Content creation
3.1 Developing content
3.2 Integrating and re-elaborating
3.3 Copyright and Licences
3.4 Programming
4. Safety
4.1 Protecting devices
4.2 Protecting data and digital identity
4.3 Protecting health
4.4 Protecting the environment
5. Problem solving
5.1 Solving technical problems
5.2 Expressing needs & identifying technological responses
5.3 Innovating, creating and solving using digital tools
5.4 Identifying digital competence gaps

8. Additional studies
2015 Ongoing
• Computational thinking
• E-Textbooks in Poland
• Learning analytics
• Policy models for digital transformation of E&T
Publisehd in 2015:
• SharedOER
• Science 2.0
Draft 2016 WP…
• Policy recommendations for opening-up in HE
• Piloting DigcompOrg
• MoocKnowledge
• Socio-economic impact of digital learning

9. http://is.jrc.ec.europa.eu/pages/EAP/eLearningPublications.html

10. JRC-IPTS project on behalf
of DG EAC | 2013-2015
Aim: To propose a framework
for opening up practices in
higher education institutions
Policy: COM 2013 Opening Up
Education
Why: To raise awareness and
provide support to higher
education institutions in the
design of strategies for opening
up education
Project main themes:
Dimensions of open education,
institutional strategies for
opening up education
OPENEDU OVERVIEW

11. OPENEDU studies
Besides the in-house research OPENEDU runs 5 studies:
Moocknowledge: a survey on MOOC learners (ongoing)
OpenCred: desk research and case studies on recognition of
non-formal learning via MOOCs (May-November 2015)
OpenSurvey: a representative survey of higher education
institutions in 5 European countries to enquire about their
openness strategies (ended Nov 2015)
OpenCases: case studies on openness in higher education
(ended Nov 2015)
BMOpen: exploring a framework for assessing & developing
business models for open education (ongoing)

12. OpenEdu Framework
Andreia Inamorato dos Santos
Yves Punie
Jonatan Castaño Muñoz
IPTS 2015

13. An OE framework: rationale
The framework is being designed to support higher education
institutions in Europe to make strategic decisions on open
education. By proposing a scope for open education and presenting its
core and transversal dimensions, the framework aims to promote
transparency and to propose a common language for open
education in Europe.

14. How has it being designed?
 drawing on studies' results: from previous
and current (designed-for-purpose) IPTS
research on open education
 drawing on results from intensive desk
research, to include grey literature
(websites, blogs, newspapers, reports etc.)
 consulting experts on the theme (1st
OpenEdu workshop in June 2014)
 consulting the academic literature to check
the appearance/validity/context of the
dimensions of open education
 validating it with experts: online
consultation with 60 experts across the
globe
QUESTIONS
Workshop with the
target audience:
YOU

15. What does the
framework look
like?
√ Open Education definition
• Dimensions:
• 6 core: access, content, pedagogy,
recognition, collaboration, technology,
research
• 4 transversal: strategy, leadership,
technology, quality
For each dimension of open
education, the framework brings:
√ Dimension definition
√ Rationale
√ Components
√ descriptors
√ Example
QUESTIONS

16. Why and how should I/my
institution use the
framework?
How:
√ as a foundation for developing
insight, inspire vision and see new
perspectives and develop new
ideas
√ as a tool to developing a position
√ by doing creative thinking on the
framework propositions
Why:
√ to challenge conventional wisdom
√ because it provides a guide to think
through critical questions. No framework
provides definitive answers. The
answers come through the insights
generated by the process
√ the framework as a box is limiting. But
throwing it out is also limiting. Without a
framework managers tend to overlook
important considerations or put great
effort into reinvesting well-understood
ideas. The trick is to use the framework
without getting trapped in it
QUESTIONS

17. Strategy is the creation of a unique and
valuable position, involving a different
set of activities
Michael Porter, 1996

18. "Plans are worthless. Planning is priceless."
Dwight David Eisenhower
Open Education
Strategic Planning
Template
√ accompanies the framework
√ is openly licensed so can be
adapted and distributed
√ helps to develop activities,
proposes a shared caused
√ once completed, provides a
direction and guidance about what
the institution will do and will not
do

19. Objectives of the workshop
For us:
√ to discuss and gain insight into all
the elements of the framework
√ to verify the usability and potential
of the framework
√ to promote an opportunity for
discussion and interaction among
university executives from different
EU Member States on the subject
of open education
And for you?
Please tell us your
name, position,
institution and country
and what you expect
from the workshop – no
more than 1 minute

20. OPEN EDUCATION DEFINITION
• Open education is a mode of realising education
enabled by digital technologies aiming to widen
access and participation to everyone. It offers
multiple ways of teaching and learning, building and
sharing knowledge, as well as a variety of access
routes to formal and non-formal education, bridging
them.
• (Source: OpenEdu IPTS, 2015 – work in progress)

21. Group discussion
• How is open education defined in your institution?
What is the value of an open education strategy?
• Does your institution have an open education
strategy? If so, what is it? If not, why is this so?

22. Plenary discussion
How do you think the definition presented is fit for
purpose?
• Open education is a mode of realising education enabled by
digital technologies aiming to widen access and participation to
everyone. It offers multiple ways of teaching and learning,
building and sharing knowledge, as well as a variety of access
routes to formal and non-formal education, bridging them.

23. CORE DIMENSIONS OF OPEN EDUCATION
1- ACCESS
2- CONTENT
3- PEDAGOGY
4- COLLABORATION
5- RECOGNITION
6- RESEARCH
(OpenEdu Project, 2015)

24. TRANSVERSAL DIMENSIONS OF OPEN EDUCATION
7- TECHNOLOGY
8- STRATEGY
9- QUALITY
10- LEADERSHIP
(OpenEdu Project, 2015)

25. Group discussion
How do these dimensions fit your
existing/future open education plan?

26. OpenCases Study
(OpenEdu and University of Bath's team, 2015)
Manuel Souto Otero
Robin Shields
Predrag Lažetić
Andreia Inamorato dos Santos
Jonatan Castaño Muñoz
Axelle Devaux
Stephanie Oberheidt
Yves Punie
Full report to be published early 2016
Webpage: http://is.jrc.ec.europa.eu/pages/EAP/opencases.html
OpenCases catalogue of mini cases on open education in Europe:
http://bit.ly/1iCTEnk

27. Case studies on openness in higher education
Aim: To understand in detail the motivations, enablers and barriers for
higher education institutions to open up education (ends June
2015)
What: 7 case studies (University of Bath:
M. Souto-Otero, R. Shields, et al.):
Case 1- OERu (International)
Case 2- TORQUE (Switzerland)
Case 3- FUN (France Université Numeriqué) (France)
Case 4- TU DELFT (Netherlands)
Case 5- Carlos III de Madrid (Spain)
Case 6- Open AGH (Poland)
Case 7- Open University of Bavaria (Germany)
OpenCases
IPTS in-house:
Case 8- ALISON
(Ireland)
Case 9- OpenUpEd
(European)

28. ● Access & education as a public good (third mission):
• To increase mainstream adoption of open education for all institutions in the world
and make HE more financially sustainable and affordable for all learners.
"There are currently over 100 million learners who demand HE but do not have
access to it". (OERu)
• To provide free online education to all, alternative to traditional education an
bridging education and work. (ALISON, social enterprise)
• Increase and widen access to HE, wider community benefitting from OE:
"Materials that are produced with public funding should be available to the public"
TUDelft [Also for visibility and reputation building]
OpenCases
WHY?
● Institutional strategies: Increasing university visibility – Following international trend –
Logical step from previous engagement in educational technology (tech. university) –
Improve on-campus education (ETH Zurich)
● Political priority (FR): Promoting OE to improve students’ learning outcomes, increase
access to lifelong learning and promote the visibility and attractiveness of French
education and training offer (FUN)

29. OpenCases
publications
• Published:
To be published early 2016 (full
cases):
OpenCases: case
studies on openness
in higher education
Link to the published report

30. MOOCKnowledge
Study
MOOCKnowledge consortium:
JRC-IPTS
OUNL
UPM
UOC

31. WHAT IS MOOCKNOWLEDGE?
It is a large scale survey aiming to get a better
understanding of the European MOOC learners

32. MOOCs are progressively adopted by EU HEI & individuals
21.8% HEIs offering, 19% planning to offer (OpenSurvey)
Evidence-based policy on OE needs to understand better the
demand side of MOOCs
RATIONALE- WHY?

33. RATIONALE-WHY?
MOOCKnowledge aims to overcome a lack of:
• Data at a European level. Most data is US centric.
• Large scale and cross- provider data. It would allow comparisons between MOOCS
(or groups of MOOCS) and the analysis of subpopulations ( e.g. teacher training,
language learners, unemployed people, migrants). Most data come from a single
MOOC or MOOC provider.
• Long-term data. It would allow the analysis of outcomes of taking a MOOC.

34. METHODS - HOW?
Survey to MOOC learners
… from different EU MOOCs- This allows data aggregation and the building of a
large scale data set.
+
… using a standardised and multilingual questionnaire. This allows the
comparison between MOOCs or groups of MOOCs.
+
… collecting data in three moments of time: Pre-MOOC, Post-MOOC and follow-
up (after 1 year), that allows to measure long-term impact.

35. RESEARCH FOCUS - WHAT?
• Population details (pre-MOOC)
• Socio-economic and demographics
• Lifelong learning profile
• ICT and (self-directed) learning skills
• Expected outcomes and motivations for enrolling (pre-MOOC)
• Satisfaction, learning experience, completion … (post-MOOC)
• Emerging research topics
• Intention – Behaviour gap (pre vs post)
• Impact of MOOC on academic and/or job career (follow up)

36. MOOCKNOWLEDGE DATA COLLECTION FIRST PILOT
Blended
learning
HandsonICT Entrepreneurship Anxiety
Management
Business
Intelligence
Genocide TOTAL
ENROLLED 1,160 1,672 12,266 16,737 Missing 1,650
Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post
VALID
RESPONSES
29 0 173 27 317 55 715 145 349 155 156 322 1739 704
%
RESPONSES
2.5% 0% 10.3% 1.6% 2..6% 0.4% 4.3% 0.9% ---- ---- 9.4% 19.5% --- ---
EXEMPLARY RESULTS- FIRST ANALYSIS OF
PRE-QUESTIONAIRE (PILOT)

37. 54.4%
45.6%
Male Female
44.8%
63.0% 61.4%
39.9%
32.4% 34.7%
55.2%
37.0% 38.6%
60.1%
67.6% 65.3%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Male Female
GENDER

38. 13.3%
24.9% 24.5% 24.0%
9.6%
3.3%
0.4%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
15-25 26-35 36-45 46-55 56-65 66-75 76-85
AGE

39. 0.3%
1.9%1.2%1.4%0.9%
8.9%
15.5%
17.5%
5.2%
13.8%
12.4%
3.4%
4.6%
4.7%
9.1%
3.5%
5.9%
5.4%
20.7%
22.9%
26.6%
28.6%
14.5%
19.3%
21.7%
75.9%
63.6%
52.8%
42.9%
75.7%
59.6%
59.6%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Blended Learning
Business Intelligence
Entrepreneurship
Genocide (Hebrew)
HandsonICT
Test Anxiety
All
Pre-primary or primary education Lower secondary or upper secondary education Post-secondary non-tertiary education
First stage of tertiary education Second stage of tertiary education
HIGHEST LEVEL OF EDUCATION

40. EMPLOYMENT

41. Employer support to Life Long Learning (LLL) activities
75%
18%
6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Encouragement Time Cost
32.09%
8.84%
59.07%
yes no Employer does not know about participation
N=1077 only workers

42. Appreciation of MOOCs by employer & Influence of certification
Mean=3.4
Mean=4.1
Mean=3.9
Mean=4.1
Mean=4.0
OVERALL MEAN=3.8
From 1 shows no appreciation at all to 7 shows very much apreciattion)
OVERALL MEAN= 4.77
From 1 has no influence at all to 8 It has very much influence
3.33
5.22 5.16
4.64
4.46
0
1
2
3
4
5
6

43. Awareness sources for MOOCs
8.69% 9.15%
34.20%
24.46%
8.80%
26.10%
4.31% 6.21% 9.10%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%

44. Past experience with MOOCs
26.2%
10.9%
11.1%
9.9%
7.2%
8.3%
4.4%
2.4%
3.3%
0.8%
5.8%
0.2%
1.6%
0.2%
0.3%
2.1%
0.2%
0.1%
0.1%
0.1%
2.6%
0.1%
0.1%
0.1%
0.1%
0.5%
1.6%
0%
5%
10%
15%
20%
25%
30%
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 More
than
25

45. Past experience with MOOCs
11.6%
0.3%
2.7%
4.3%
8.2%
11.5%
4.9%
6.3%
7.4%
3.8%
0.6%
38.5%
32.2%
0.6%
3.5%
4.3%
8.0%
8.9%
3.2%
4.1%
5.2%
3.3%
0.6%
25.9%
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
0% From 1-
10%
From 11-
20%
From 21-
30%
From 31-
40%
From 41-
50%
From 51-
60%
From 61-
70%
From 71-
80%
From 81-
90%
From 91-
99%
100%
% of completed MOOCs in the past % of completed with certificate in the past
N=1243 only those who ever enrolled in a MOOC

46. JOIN US!!!
Are you offering or planning to offer MOOCs?
We are looking for cooperation agreements
WHY?
Use of a EU common questionnaire.
Benchmarking for providers
Fast data sharing of own data with provider.
Contribution to an EU dataset (Anonymised).
Combination with local learning analytics data

47. OpenCred study
OpenEdu and University of Leicester's team
Gabi Witthaus
Andreia Inamorato dos Santos
Mark Childs
Anne-Christin Tannhäuser
Grainne Conole
Bernard Nkuyubwatsi
Yves Punie
Webpage: http://is.jrc.ec.europa.eu/pages/OpenCred/ISUNITWEBSITE-IPTS-JRC-EC.htm
Paper "An assessment-recognition matrix for analysing institutional practices in the recognition
of open learning http://bit.ly/1QRlbQK (OpenCred Phase I)

48. OpenCred
Results Phase I
1. There are varying degrees of formality of recognition
2. Factors that have the greatest impact on formality of
recognition are:
• Robustness of assessment
• Affordability of assessment for learners
• Learners’ eligibility for assessment
√ A study based on desk
research and interviews
√ Phase I completed in
11/2014. Phase II completed
in 11/2015
√ Report in the process of
being edited for publication
Paper published in the e-
Learning Papers issue 40
Outcome
A typology of institutional
practices for the Recognition
of Open Learning in Europe

49. OpenCred Diamond
0
1
2
3
4
Formality of
recognition
Affordability for
learner
Robustness of
assessment
Eligibility for
assessment/recogniti
on
OpenCred Diamond Phase I

50. OpenCred findings
Typical MOOC with little or no
recognition
E.g. CARNET (Croatia) MOOC on
Developing Courses in Moodle
Phase I

51. Other diamond shapes
MOOC with recognition for formally
enrolled students
e.g. University of Nicosia MOOC on
Digital Currencies
0
1
2
3
4
Formality of recognition
Affordability for learner
Robustness of
assessment
Eligibility for
assessment/recognition
0
1
2
3
4
Formality of recognition
Affordability for learner
Robustness of
assessment
Eligibility for
assessment/recognition
Freemium-model MOOC
e.g. University of Osnabrück
MOOC on Data Structures &
Algorithms
Phase I

52. OpenCred Conclusions
• Robust assessment is central to recognition
– Institutions either pass on the cost to learners or restrict
eligibility.
• To date recognition is only partial – no whole degrees yet
• Online education and assessment still seen by many as less
rigorous
• On-site exams with identity validation and real-time
supervision are seen as being most robust form of assessment
• ECTS credits are not yet a widely accepted currency for
recognition of open learning
Phase I

53. OpenCred Phase II
six elements of MOOC provision appear to be central to facilitating future recognition by other HEIs or
employers:
 Identity verification of the learner
 Suitable supervised assessment
 Informative credentials such as (digital) certificates or online badges that acknowledge learning
 Quality assurance
 Award of credit points
 Partnerships and collaboration with potentially “recognizing” institutions or bodies
These elements are represented in the OpenCred “traffic light model” as follows:
OpenCred, 2015
In phase II of the study
OpenCred has developed a
model with more elements:

54. OpenCred's traffic
light model
OpenCred, 2015

55. Thank you for your attention!
andreia_inamorato_dos.santos@ec.europa.eu
yves.punie@ec.europa.eu
jonatan.castano-munoz@ec.europa.eu