Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Massive Open Online Course designed to support the
development of Virtual Mobility Transversal Skills:
preliminary evalu...
Background
• Despite the growing acknowledged of Virtual Mobility, only a few researches have
investigated the impact of V...
The goal of the research
The OpenVM MOOC design was developed by all the partners through the
coordination of the Roma Tre...
Pilot phase - methodology
Our pilot-phase was carried out in order to assess the quality of the OpenVM
MOOCs and its main ...
The MOOC assessment questionnaire
https://docs.google.com/forms/d/1OJD62k4IveuUbDeFo-sw_sO4YU52gsoFyWSf1GsIkK0/edit#respon...
Sample
Characteristics – first pilot cycle
716 (F = 498; M = 215; Not
specified = 3) participants took
part in the pilot-p...
MOOC attendance
27% took part and
assessed the MOOC Active Self-
regulated Learning
22% Media and Digital Literacy
15% “Op...
Affiliation
Roma Tre
University
Beuth University
of Applied
Sciences
Universitatea
Politehnica
Timisoara
Universitat de
le...
General evaluation of the MOOCs -
Positive statements
Average N SD
Networked learning 3,5945 50 1,02593
Media and Digital ...
Badge evaluation - Positive
statements
Average N SD
Active Self-Regulated Learning 3,5351 194 0,78093
Networked learning 3...
Gamification features
evaluation - Positive
statements
Average N SD
Networked learning 3,18 50 0,94868
Collaborative learn...
Technical features evaluation -
Positive statements
Averag
e N SD
Open Education and Virtual Mobility 3,5704 27 0,66957
Ne...
Conclusive remarks on the first part of the
questionnaire
From a general overview we
can conclude that four
MOOCs at the m...
92% of participants took part in the foundation level, 51,6% in the
intermediate level and 30,6% in the advanced level in ...
This picture shows a comparison of the average
answers given for foundation, intermediate and
advanced level.
For each sta...
E-portfolio and peer assessment
activity
Average N SD
Networked learning 3 17 1,1607
Media and Digital Literacy 3,4943 58 ...
Qualitative research - first pilot
(students opinion)
Examples of qualitative answers analysed through
thematic analysis (...
Sustainability - follow up suggestions
Create resources for teachers on how to integrate the OpenVM MOOCs in their courses...
References
• Amirault, R. J., & Visser, Y. L. (2010). The impact of e-learning programs on the internationalization of the...
Upcoming SlideShare
Loading in …5
×

[EADTU & Open VM Webinar] A Massive Open Online Course Designed to Support the Development of Virtual Mobility Transversal Skills: Preliminary Evaluation Results from European Participants

64 views

Published on

Presentation by Antonella Poce and Francesca Amenduni during Webinar EADTU & Open VM on Friday 19 June 2020

Published in: Education
  • Be the first to comment

  • Be the first to like this

[EADTU & Open VM Webinar] A Massive Open Online Course Designed to Support the Development of Virtual Mobility Transversal Skills: Preliminary Evaluation Results from European Participants

  1. 1. A Massive Open Online Course designed to support the development of Virtual Mobility Transversal Skills: preliminary evaluation results from European participants Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources. Antonella Poce, Francesca Amenduni, Carlo De Medio, Maria Rosaria Re, Mara Valente, Francesco Agrusti.
  2. 2. Background • Despite the growing acknowledged of Virtual Mobility, only a few researches have investigated the impact of Virtual Mobility initiatives on participants, and most of them includes small scale studies (Hilliard, 2004; Frydenberg, & Andone, 2010; Costa, & Balula, 2014); • Results of the previous experience suggest the critical role of participants’ transversal skills in Virtual Mobility Experience (Poulová, Černá &Svobodová, 2009); • MOOCs are now being considered and applied by many institutions around the world as a valid internationalization instrument (Knight, 2014). However, Amirault and Visser (2010) show that virtual program offerings do not automatically cross borders, nor result in the same effects everywhere. Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  3. 3. The goal of the research The OpenVM MOOC design was developed by all the partners through the coordination of the Roma Tre Team. The OpenVM is aimed at developing those transverse skills necessary to be successfully involved in a Virtual Mobility programme in an international digital learning environment. Our preliminary research questions are: • To which extent students enjoyed the OpenVM MOOC design and its main components? • Are there any differences in the assessment of the 8 miniMOOCs and subMOOCs? • Does the MOOC support students’ self-regulated learning? Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  4. 4. Pilot phase - methodology Our pilot-phase was carried out in order to assess the quality of the OpenVM MOOCs and its main components: - Matching tool and group formation (Output 3); - E-assessment, more specifically quizzes, e-portfolio and peer-assessment (Output 4); - Gamification and badges (Output 5); - Learning materials and course structures (Output 6), The pilot phase is organized in three iterations designed according to the principles of the DBR and ADDIE (Output 7): 1. a pre-pilot phase (december 2018- january 2019) 2. the first pilot-phase cycle (september - december 2019) and 3. the second- pilot phase cycle (2020). Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  5. 5. The MOOC assessment questionnaire https://docs.google.com/forms/d/1OJD62k4IveuUbDeFo-sw_sO4YU52gsoFyWSf1GsIkK0/edit#responses Participants were required to express their level of agreement with a set of statements related to specific MOOC design elements on a Likert scale from 1 (strongly disagree) to 5 (totally agree). • Personal details: age, gender, affiliation and role. In this section participants are required to say which of the eight mini-MOOCs they are assessing; • Questions regarding the overall MOOC design: learning experience, quality of content instruction and support for learning; • Questions regarding digital credentials and meaningful gamification: quality of design, motivation, engagement and possibilities of choice; • Questions regarding technical aspects: use and usability; • Questions regarding the foundation level of a mini-MOOC: duration, language, content, use of multimedia; • Questions regarding the intermediate level of a mini-MOOC: extending questions from the foundation level by questions related to the matching tool and group formation activity, which are specific design elements used at this level; • Questions regarding the advanced level of a mini-MOOC: extending questions from the foundation and intermediate levels by questions related to e-portfolio and peer-assessment activities, which are specific design elements at this level; • Questions related to the investigation about the extent to which MOOCs supported self-regulated learning. In this section participants are required to answer also to open-ended questions.
  6. 6. Sample Characteristics – first pilot cycle 716 (F = 498; M = 215; Not specified = 3) participants took part in the pilot-phase. Most of the participants were university students and only 14 teachers participated in the survey. Gender Number % • Female 498 69,6% • Male 215 30% • Prefer Not to say 3 4% Role • University student 699 97,6% • Teacher 14 2% • Other 3 0,4% Age • Less than 20 153 21,4 • Between 21 and 23 348 48,6 • Between 24 and 26 100 14,0 • Between 27 and 30 44 6,1 • Between 31 and 35 34 4,7 • More than 36 37 5,2 Total 716 100% Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  7. 7. MOOC attendance 27% took part and assessed the MOOC Active Self- regulated Learning 22% Media and Digital Literacy 15% “Open-mindedness” 14%“Intercultural Skills” 7% “Networked Learning” 6%“Autonomy-drive learning” 5%“Collaborative Learning” and “Open Education and Virtual Mobility” Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  8. 8. Affiliation Roma Tre University Beuth University of Applied Sciences Universitatea Politehnica Timisoara Universitat de les Illes Balears (UIB) AUNEGE Open Universiteit – Welten Instituut External institutions Total MOOC Media and Digital Literacy 0 74 61 1 11 1 7 155 Intercultural Skills 6 1 76 1 3 2 13 102 Active Self-Regulated Learning 132 24 17 21 0 0 0 194 Autonomy drive-learning 4 0 34 1 0 1 0 40 Networked learning 1 6 40 0 0 0 3 50 Collaborative learning 2 1 36 0 0 0 0 39 Open-mindedness 5 2 100 1 1 0 0 109 Open Education and Virtual Mobility 8 9 4 4 0 0 2 27 Total 158 117 368 29 15 4 25 716 MOOCs most attended by each partner institution Disclaimer: The creation of these resources has been (partially) funded by the ERASMUS+ grant program of the European Union under grant no. 2017-1-DE01-KA203-003494. Neither the European Commission nor the project's national funding agency DAAD are responsible for the content or liable for any losses or damage resulting of the use of these resources.
  9. 9. General evaluation of the MOOCs - Positive statements Average N SD Networked learning 3,5945 50 1,02593 Media and Digital Literacy 3,6938 155 0,65947 Collaborative learning 3,7203 39 0,72619 Active Self-Regulated Learning 3,7381 194 0,6769 Open-Education and Virtual Mobility 3,7542 27 0,61802 Average 3,7689 716 0,72937 Open-mindedness 3,8182 109 0,769 Autonomy driven-learning 3,8477 40 0,72797 Intercultural Skills 3,9661 102 0,71558 The general evaluation of the MOOCs was quite positive. The average score for each MOOC is always higher than 3,5 out of 5 points (we used a Likert Scale from 1 to 5, where the median is 3). The total average was 3,77. Three MOOCs out of eight obtained scores higher than the average: 1. Open-mindedness (109 answers) 2. Autonomy-driven learning (40 answers) 3. Intercultural skills (102 answers).
  10. 10. Badge evaluation - Positive statements Average N SD Active Self-Regulated Learning 3,5351 194 0,78093 Networked learning 3,56 50 1,06904 Media and Digital Literacy 3,591 155 0,84741 Intercultural Skills 3,6922 102 0,76853 Open-mindedness 3,7193 109 0,84989 Collaborative learning 3,7385 39 0,83874 Autonomy driven-learning 3,93 40 0,72402 The general evaluation of the badges was quite positive. The average score for badge is always higher than 3,5 out of 5 points (we used a Likert Scale from 1 to 5, where the median is 3). Please consider that we do not have badges for the MOOC “Open-Education and Virtual Mobility”. Participants expressed a preference for badges in three MOOCs: - Open-mindedness - Collaborative learning - Autonomy driven-learning
  11. 11. Gamification features evaluation - Positive statements Average N SD Networked learning 3,18 50 0,94868 Collaborative learning 3,2487 39 0,69995 Media and Digital Literacy 3,2613 155 0,66189 Active Self-Regulated Learning 3,2964 194 0,62216 Open Education and Virtual Mobility 3,3148 27 0,6125 Average 3,3226 716 0,68037 Open-mindedness 3,3917 109 0,69949 Autonomy driven-learning 3,41 40 0,68605 Intercultural Skills 3,4578 102 0,63315 The general evaluation of the gamification features was still more than 3 but lower than 3,5. Please consider that we do not have badges for the MOOC “Open-Education and Virtual Mobility”. The total average was 3,32 Participants expressed a preference for gamification in three MOOCs: - Open-mindedness - Autonomy driven-learning - Intercultural skills
  12. 12. Technical features evaluation - Positive statements Averag e N SD Open Education and Virtual Mobility 3,5704 27 0,66957 Networked learning 3,6449 49 1,09412 Media and Digital Literacy 3,6768 151 0,8339 Active Self-Regulated Learning 3,7403 191 0,71642 Average 3,7946 704 0,8078 Collaborative learning 3,8378 37 0,78717 Autonomy driven-learning 3,88 40 0,77499 Open-mindedness 3,9 108 0,83184 Intercultural Skills 4,0436 101 0,74222 The technical features evaluation of the MOOCs was quite positive. The average score for each MOOC is always higher than 3,5 out of 5 points (we used a Likert Scale from 1 to 5, where the median is 3). The total average was 3,79. Four MOOCs out of eight obtained scores higher than the average: 1. Collaborative learning 2. Autonomy-driven learning 3. Open-mindedness 4. Intercultural skills
  13. 13. Conclusive remarks on the first part of the questionnaire From a general overview we can conclude that four MOOCs at the moment were the most appreciated considering 1. general features 2. badges 3. gamification 4. technical features 1. Collaborative learning 2. Autonomy-driven learning 3. Open-mindedness 4. Intercultural skills
  14. 14. 92% of participants took part in the foundation level, 51,6% in the intermediate level and 30,6% in the advanced level in one of the 8 miniMOOCs. Which level did participants attend?
  15. 15. This picture shows a comparison of the average answers given for foundation, intermediate and advanced level. For each statement, in all the levels average scores are higher than 3,55. We can see that course duration is considered better in foundation level than intermediate and advanced level. Usually foundation level achieved higher average scores. However, multimedia contents, and language are more appreciated in advanced level.
  16. 16. E-portfolio and peer assessment activity Average N SD Networked learning 3 17 1,1607 Media and Digital Literacy 3,4943 58 0,84982 Active Self-Regulated Learning 3,6 30 0,74997 Open-mindedness 3,6471 51 1,05496 Collaborative learning 3,6923 13 0,93751 Autonomy drive-learning 3,9167 12 0,76706 Intercultural Skills 3,9394 33 0,85576 Group formation and matching tool Average N SD Networked learning 3,5486 24 1,39658 Media and Digital Literacy 3,9627 85 0,99116 Active Self-Regulated Learning 3,966 54 1,03926 Collaborative learning 4,1587 21 1,07945 Open-mindedness 4,1689 73 1,11958 Intercultural Skills 4,2091 55 1,22484 Autonomy drive-learning 4,2826 23 0,93802 The general evaluation of - e-portfolio and peer-assessment - group formation and matching tool was quite positive. The average score for Group Formation is always higher than 3,5 out of 5 points; The average score for E-portfolio and peer- assessment is lower than 3,5 only for 1. Networked Learning 2. Media and Digital Literacy. Participants expressed a preference for e- assessment and group formation in: - Open-mindedness - Collaborative learning - Autonomy drive-learning
  17. 17. Qualitative research - first pilot (students opinion) Examples of qualitative answers analysed through thematic analysis (N=20) I enjoyed doing the Course need more explanation The Foundation Level MOOC was pretty short and that took an influence on my answers it helped me a little bit self observation It helped me to use new Technologies to learn and to exercise my brain Thematic analysis Helpful to improve strategy of study: 14 SRL skills development: 10 General Enjoyment: 9 Generally Useful/Helpful: 9 Refers to technologies: 5 Negative comments: 3 Not categorized: 2
  18. 18. Sustainability - follow up suggestions Create resources for teachers on how to integrate the OpenVM MOOCs in their courses Different OERs for teachers and students could be created taking into account: - How to create groups (for teachers) - How to work a good e-portfolio out (for teachers and students) - How to support students’ peer assessment activities (both for teachers and students) These OERs should be based on the results of pilot experience.
  19. 19. References • Amirault, R. J., & Visser, Y. L. (2010). The impact of e-learning programs on the internationalization of the university. New York, NY: Nova Science Publishers. • Bates, A. (2015). Variations in MOOC designs. BC Open Texbooks. • Buchem, I., Tur, G. & Urbina, S. (2018). Quality assurance for attainment, assessment and recognition of virtual mobility skills in context of open education. QA Framework in the Open Virtual Mobility project. Paper presented at Edulearn Conference 2-4 July 2018. DOI: 10.21125/edulearn.2018.0318 Retrieved from https://iated.org/concrete3/view_abstract.php?paper_id=65036 • Costa, N., & Balula, A. (2014). Virtual mobility and learning for PhD students of six European countries-students' programme evaluation. Profesinis rengimas: tyrimai ir realijos, 2014, nr. 25, p. 178-188. • Daniel, J. (2012). Making sense of MOOCs: Musings in a maze of myth, paradox and possibility. Journal of interactive Media in education, 2012(3). DOI: http://doi.org/10.5334/2012-18 • Frydenberg, M., & Andone, D. (2010). Two Screens and an Ocean: Collaborating across Continents and Cultures with Web-Based Tools. Information Systems Education Journal, 8(55), n55. • Hilliard, A. (2004, January). Outline and Evaluation of a Joint European and Canadian Virtual Mobility: e-Learning Project. In ECEL2006-5th European Conference on e-Learning: ECEL2006(p. 163). Academic Conferences Limited. • Knight, J. (2014). Three generations of crossborder higher education: New developments, issues and • Konert, J., Burlak, D., & Steinmetz, R. (2014, September). The group formation problem: an algorithmic approach to learning group formation. In European Conference on Technology Enhanced Learning (pp. 221-234). Springer, Cham. • Poulová, P. E. T. R. A., Černá, M. I. L. O. S. L. A. V. A., & Svobodová, L. (2009). University Network? Efficiency of Virtual Mobility. In Proceedings of the 5th WSEAS/IASME International Conference on EDUCATIONAL TECHNOLOFIES (EDUTE09) (pp. 87-92). • Rajagopal, K., & Firssova, O. (2018, April). Virtual Mobility in the context of Open Education: re-establishing the boundaries. In Open Education Global Conference 2018: Transforming education through open approaches. Routledge handbook of international education and development (pp. 340-358). Abingdon: Routledge. • Teichler, U. (2004). The changing debate on internationalization of higher education. Higher Education, 48, 5-26. • Teresevičienė M., Volungevičienė A., and Daukšienė E. (Eds.) (2011). Virtual mobillity for Teachers and Students in Higher Education. Kaunas: Vytautas Magnus University. • Yuan, L., & Powell, S. J. (2013). MOOCs and open education: Implications for higher education. Retrieved from: https://e-space.mmu.ac.uk/619735/1/MOOCs-and-Open-Education.pdf • Zimmerman, B. J. (2013). Theories of self-regulated learning and academic achievement: An overview and analysis. In Self-regulated learning and academic achievement (pp. 10-45). Routledge.

×