SlideShare a Scribd company logo
DEVELOPING A MOOC EXPERIMENTATION
PLATFORM: INSIGHTS FROM A USER STUDY
The University of Edinburgh
Vitomir Kovanović, Srećko Joksimović, Philip Katerinopoulos,
Charalampos Michail, George Siemens, & Dragan Gašević
Simon Fraser University,
Vancouver, Canada
Lak17
@vkovanovic
v.kovanovic@ed.ac.uk
http://vitomir.kovanovic.info
WHY ARE MOOCS INTERESTING (FOR A RESEARCHER)
MOOCs
2
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
1. Educational researchers historically small scale – “Big Data” in education
2. Potential for improving on-campus learning experience
3. Innovate for teaching innovations (e.g., flipped classroom models)
4. Great potential for understanding human learning
5. Experiment with different instructional interventions
WHY ARE MOOCS INTERESTING (FOR A RESEARCHER)
MOOCs
3
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
1. Educational researchers historically small scale – “Big Data” in education
2. Potential for improving on-campus learning experience
3. Innovate for teaching innovations (e.g., flipped classroom models)
4. Great potential for understanding human learning
5. Experiment with different instructional interventions
ALSO (of course)
1. Democratization of learning
2. Helping students without access to good quality education
3. Closing the achievement gap
4. Helping developing countries
5. Reduce costs of education
6. …..
CURRENT RESEARCH CHALLENGES
MOOCs
4
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
CURRENT RESEARCH CHALLENGES
MOOCs
5
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
CURRENT RESEARCH CHALLENGES
MOOCs
6
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
Data is notoriously had to collect and analyze
1. Diverse data sources
2. Tracking users across platforms (e.g., twitter, surveys)
3. Different data formats (CSV formats, JSON formats)
4. Disconnected data sources (_general, _mappings, _forum)
5. Anonymization
6. IRBs
7. Data analysis requires understanding of the data science methods (e.g., feature engineering)
8. Understanding of the pedagogical domain
9. Correctly interpreting the data
10. Issues with replication of educational studies
11. No ad-hoc experimentation, A/B testing must be planned far in advance
CURRENT RESEARCH CHALLENGES
MOOCs
7
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
1. Most MOOC instructors lack these skills
1. One reason for why Computer Science researches dominated MOOC research
GOAL
MOOCs
8
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
Develop a tool that will assist MOOC instructors/researchers in analyzing the MOOC data
Guiding principles:
1. No technical prerequisites
1. No database installs
2. Incremental data import
1. As the new data dump becomes available, it should be possible to import it.
3. Support cluster analysis
1. We will add additional types of analysis over time (e.g., SNA)
4. Enable follow-up analysis
1. Details about interventions should be possible to analyze in R/SPSS
5. Support class intervention
1. Instructors should be able to send emails to selected groups of students
6. Support study replication
1. Support pre-existing analysis within the tool
GOAL
MOOCs
9
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
Develop a tool that will assist MOOC instructors/researchers in analyzing the MOOC data
Design decisions:
1. For now, desktop based
1. Later maybe provide a web-based version
2. Data sharing & IRB issues
2. No fancy database scheme abstractions, focus on one platform at a time
1. Coursera session-based implemented, edX in progress
3. Java-based to support multiple platforms
OVERVIEW TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
10
OVERVIEW TAB
MOOCito
11
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
DETAILS TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
12
ANALYZE TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
13
ANALYZE | INDICATORS TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
14
ANALYZE | RUN ANALYSIS TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
15
ANALYZE | EVALUATE TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
16
ANALYZE | EVALUATE TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
17
INTERVENE TAB
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
18
EVALUATION
MOOCito
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
19
1. Conducted a user study with 11 participants (UK, USA, South America)
1. All participants were experienced with online learning, MOOCs, and Coursera platform
2. Case scenario of a hypothetical MOOC instructor halfway through the course
3. Survey questionnaire based on technology acceptance model (TAM)
STUDY
MOOCito
20
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
RESULTS
MOOCito
1. Overall, participants were satisfied with the proposed system and its perceived usability and
ease-of-use.
2. Most positive responses:
1. The ability to generate interesting questions and hypotheses (4.73),
2. The ability of the tool to provide insights into students’ engagement with the learning
environment (4.64), and
3. The capacity to provide different interventions to subgroups of students (4.55).
3. In open-ended responses, participants emphasized:
1. The selection of engagement indicators, weekly plots, and clear interface.
2. The flexibility and ease of performing interventions without “the dangers of providing
blanket information inappropriately” to a particular target subset of students
3. The use of templates and “variables” in the message body.
4. Strong willingness (4.36) to use MOOCito and would like to be able to use it in their courses
(4.00).
1. Willingness to use it for research was higher (4.55) than for course design and teaching
purposes (4.30).
21
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
ISSUES
MOOCito
1. Concerns regarding the accessibility of the tool for “typical” MOOC instructors.
1. Not familiar with: 1. Terminology, 2. Engagement indicators, 3. Statistical analyses, or 4.
Study “templates”
2. Future work: 1. Short summaries, 2. Video tutorials and 3. Bibliographical information
2. What would be the best way to analyze the data?
1. Future work: Additional information on the “standard approaches”
3. Simplification of the analysis steps (D4)
1. Future work: More streamlined analysis, better guidelines and support for instructors
with little background and experience in statistics.
4. Instructors should be able to go from indicators of engagement to the actual student
content.
5. Challenge of knowing when or how to intervene,
1. The need for more “interpretable” results, summarized in a form which is easy to act
upon.
2. Simple descriptions using “low,” “moderate,” and “high” for describing engagement
22
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
PROGRESS
MOOCito
1. Support for edX and Coursera rolling session format
2. Work on user interface
3. Work on SNA analysis
4. Modularizing it so it is extensible
5. “Interpretable results”
23
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
SUMMARY
MOOCito
“This has the potential to be a powerful tool in educational research as well
as a means of tracking learner engagement day-to-day.”.
24
Lak17Kovanovic et al. – Developing a MOOC experimentation platform
THANK YOU
QUESTIONS?
Web: http://vitomir.kovanovic.info
Mail: v.kovanovic@ed.ac.uk
Twitter: @vkovanovic
LUNCH IS GETTING COLD
The end
25
Lak17Kovanovic et al. – Developing a MOOC experimentation platform

More Related Content

What's hot

Analysis of social interactions and prediction of assignment grades in a Mass...
Analysis of social interactions and prediction of assignment grades in a Mass...Analysis of social interactions and prediction of assignment grades in a Mass...
Analysis of social interactions and prediction of assignment grades in a Mass...
eMadrid network
 
Inquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & OpportunitiesInquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & Opportunities
Vitomir Kovanovic
 
MOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunitiesMOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunities
Vitomir Kovanovic
 
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
Christian M. Stracke
 
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
EDEN Digital Learning Europe
 
What does effective online/blended teaching look like?
What does effective online/blended teaching look like?What does effective online/blended teaching look like?
What does effective online/blended teaching look like?
Vitomir Kovanovic
 
2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results
Christian M. Stracke
 
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
Christian M. Stracke
 
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
Christian M. Stracke
 
Deep reinforcement learning
Deep reinforcement learningDeep reinforcement learning
Deep reinforcement learning
SyedMahmoodAliRoomi
 
OpenCourseWorld - Why and how to MOOC
OpenCourseWorld - Why and how to MOOCOpenCourseWorld - Why and how to MOOC
OpenCourseWorld - Why and how to MOOC
OpenCourseWorld
 
The current state of accessibility of MOOCs: What are the next steps?
The current state of accessibility of MOOCs: What are the next steps?The current state of accessibility of MOOCs: What are the next steps?
The current state of accessibility of MOOCs: What are the next steps?
Global OER Graduate Network
 
EDENRW8: To MOOC or not to MOOC that is the Question
EDENRW8: To MOOC or not to MOOC that is the QuestionEDENRW8: To MOOC or not to MOOC that is the Question
EDENRW8: To MOOC or not to MOOC that is the Question
Willem van Valkenburg
 
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
Brandon Muramatsu
 
An international collaboration in the design experience of a MOOC series
An international collaboration in the design experience of a MOOC seriesAn international collaboration in the design experience of a MOOC series
An international collaboration in the design experience of a MOOC series
Open Education Global (OEGlobal)
 
SFU SIAT Comprehensive Examination
SFU SIAT Comprehensive ExaminationSFU SIAT Comprehensive Examination
SFU SIAT Comprehensive Examination
Vitomir Kovanovic
 
Open Education: the MOOC Experience
Open Education: the MOOC ExperienceOpen Education: the MOOC Experience
Open Education: the MOOC Experience
Rémi Bachelet
 
Hybrid Courses & Curriculum
Hybrid Courses & CurriculumHybrid Courses & Curriculum
Hybrid Courses & Curriculum
Christopher Rice
 
MOOCs and personal learning: reality or myth?
MOOCs and personal learning: reality or myth?MOOCs and personal learning: reality or myth?
MOOCs and personal learning: reality or myth?
Inge de Waard
 
Kraków Revisited
Kraków RevisitedKraków Revisited
Kraków Revisited
Global OER Graduate Network
 

What's hot (20)

Analysis of social interactions and prediction of assignment grades in a Mass...
Analysis of social interactions and prediction of assignment grades in a Mass...Analysis of social interactions and prediction of assignment grades in a Mass...
Analysis of social interactions and prediction of assignment grades in a Mass...
 
Inquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & OpportunitiesInquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & Opportunities
 
MOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunitiesMOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunities
 
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
2018-07-13 MOOQ Conference in Athens MOOQ and the Quality of MOOCs - How it s...
 
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
Assessing quality in an Open Virtual Mobility MOOC, #edlw2019
 
What does effective online/blended teaching look like?
What does effective online/blended teaching look like?What does effective online/blended teaching look like?
What does effective online/blended teaching look like?
 
2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results2017-08-30 EARLI MOOQ Interactive Workshop Results
2017-08-30 EARLI MOOQ Interactive Workshop Results
 
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
2017-08-30 EARLI MOOQ Interactive Workshop Presentation Stracke
 
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
2016-12-13 DRDC Conference MOOQ Workshop Christian M. Stracke
 
Deep reinforcement learning
Deep reinforcement learningDeep reinforcement learning
Deep reinforcement learning
 
OpenCourseWorld - Why and how to MOOC
OpenCourseWorld - Why and how to MOOCOpenCourseWorld - Why and how to MOOC
OpenCourseWorld - Why and how to MOOC
 
The current state of accessibility of MOOCs: What are the next steps?
The current state of accessibility of MOOCs: What are the next steps?The current state of accessibility of MOOCs: What are the next steps?
The current state of accessibility of MOOCs: What are the next steps?
 
EDENRW8: To MOOC or not to MOOC that is the Question
EDENRW8: To MOOC or not to MOOC that is the QuestionEDENRW8: To MOOC or not to MOOC that is the Question
EDENRW8: To MOOC or not to MOOC that is the Question
 
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
Project Greenfield: A New Way of thinking about OpenCourseWare and Open Educa...
 
An international collaboration in the design experience of a MOOC series
An international collaboration in the design experience of a MOOC seriesAn international collaboration in the design experience of a MOOC series
An international collaboration in the design experience of a MOOC series
 
SFU SIAT Comprehensive Examination
SFU SIAT Comprehensive ExaminationSFU SIAT Comprehensive Examination
SFU SIAT Comprehensive Examination
 
Open Education: the MOOC Experience
Open Education: the MOOC ExperienceOpen Education: the MOOC Experience
Open Education: the MOOC Experience
 
Hybrid Courses & Curriculum
Hybrid Courses & CurriculumHybrid Courses & Curriculum
Hybrid Courses & Curriculum
 
MOOCs and personal learning: reality or myth?
MOOCs and personal learning: reality or myth?MOOCs and personal learning: reality or myth?
MOOCs and personal learning: reality or myth?
 
Kraków Revisited
Kraków RevisitedKraków Revisited
Kraków Revisited
 

Viewers also liked

Penetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimationPenetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimation
Vitomir Kovanovic
 
Learning Analytics for Communities of Inquiry
Learning Analytics for Communities of InquiryLearning Analytics for Communities of Inquiry
Learning Analytics for Communities of Inquiry
Vitomir Kovanovic
 
Automated Content Analysis of Discussion Transcripts
Automated Content Analysis of Discussion TranscriptsAutomated Content Analysis of Discussion Transcripts
Automated Content Analysis of Discussion Transcripts
Vitomir Kovanovic
 
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
Vitomir Kovanovic
 
Automated content analysis of cognitive presence: Improving the quality of in...
Automated content analysis of cognitive presence: Improving the quality of in...Automated content analysis of cognitive presence: Improving the quality of in...
Automated content analysis of cognitive presence: Improving the quality of in...
Vitomir Kovanovic
 
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
Vitomir Kovanovic
 
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 TutorialTopic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
Vitomir Kovanovic
 

Viewers also liked (7)

Penetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimationPenetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimation
 
Learning Analytics for Communities of Inquiry
Learning Analytics for Communities of InquiryLearning Analytics for Communities of Inquiry
Learning Analytics for Communities of Inquiry
 
Automated Content Analysis of Discussion Transcripts
Automated Content Analysis of Discussion TranscriptsAutomated Content Analysis of Discussion Transcripts
Automated Content Analysis of Discussion Transcripts
 
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
 
Automated content analysis of cognitive presence: Improving the quality of in...
Automated content analysis of cognitive presence: Improving the quality of in...Automated content analysis of cognitive presence: Improving the quality of in...
Automated content analysis of cognitive presence: Improving the quality of in...
 
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
Towards Automated Classification of Discussion Transcripts: A Cognitive Prese...
 
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 TutorialTopic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
 

Similar to Kovanović et al. 2017 - developing a mooc experimentation platform: insights from a user study

EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EUmoocs
 
Learning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCsLearning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCs
Rebecca Ferguson
 
Making MOOCs and changing open educational practices
Making MOOCs and changing open educational practicesMaking MOOCs and changing open educational practices
Making MOOCs and changing open educational practices
ROER4D
 
«Innovations in pedagogy using MOOCs» / Ting-Chuen Pong, professor of Comput...
«Innovations in pedagogy using MOOCs» /  Ting-Chuen Pong, professor of Comput...«Innovations in pedagogy using MOOCs» /  Ting-Chuen Pong, professor of Comput...
«Innovations in pedagogy using MOOCs» / Ting-Chuen Pong, professor of Comput...
eMadrid network
 
Making MOOCs and changing open educational practices
Making MOOCs and changing open educational practicesMaking MOOCs and changing open educational practices
Making MOOCs and changing open educational practices
Open Education Global (OEGlobal)
 
From OpenCourseWare to Open CourseWare
From OpenCourseWare to Open CourseWareFrom OpenCourseWare to Open CourseWare
From OpenCourseWare to Open CourseWare
Brandon Muramatsu
 
How to MOOC? – A pedagogical guideline for practitioners
How to MOOC? – A pedagogical guideline for practitionersHow to MOOC? – A pedagogical guideline for practitioners
How to MOOC? – A pedagogical guideline for practitioners
Martin Ebner
 
Online learning for MOOC team developers
Online learning for MOOC team developersOnline learning for MOOC team developers
Online learning for MOOC team developers
Nelson Jorge
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analytics
Rebecca Ferguson
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EUmoocs
 
Df e rr355_-_opportunities_for_moo_cs_in_schools_final
Df e rr355_-_opportunities_for_moo_cs_in_schools_finalDf e rr355_-_opportunities_for_moo_cs_in_schools_final
Df e rr355_-_opportunities_for_moo_cs_in_schools_final
Ian Koxvold
 
Supporting educators as designers of complex blended learning scenarios: visu...
Supporting educators as designers of complex blended learning scenarios: visu...Supporting educators as designers of complex blended learning scenarios: visu...
Supporting educators as designers of complex blended learning scenarios: visu...
Laia Albó
 
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
eMadrid network
 
Learning Analytics: Realizing their Promise in the California State University
Learning Analytics:  Realizing their Promise in the California State UniversityLearning Analytics:  Realizing their Promise in the California State University
Learning Analytics: Realizing their Promise in the California State University
John Whitmer, Ed.D.
 
Learning Analytics for MOOCs: EMMA case
Learning Analytics for MOOCs: EMMA caseLearning Analytics for MOOCs: EMMA case
Moocs
MoocsMoocs
Moocs
COHERE2012
 
Some Issues Affecting the Sustainability of Open Learning Courses
Some Issues Affecting the Sustainability of Open Learning Courses Some Issues Affecting the Sustainability of Open Learning Courses
Some Issues Affecting the Sustainability of Open Learning Courses
James Aczel
 
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
eMadrid network
 
Qmm sept 22 2015
Qmm sept 22 2015Qmm sept 22 2015
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
Christian M. Stracke
 

Similar to Kovanović et al. 2017 - developing a mooc experimentation platform: insights from a user study (20)

EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
 
Learning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCsLearning analytics, learning design and MOOCs
Learning analytics, learning design and MOOCs
 
Making MOOCs and changing open educational practices
Making MOOCs and changing open educational practicesMaking MOOCs and changing open educational practices
Making MOOCs and changing open educational practices
 
«Innovations in pedagogy using MOOCs» / Ting-Chuen Pong, professor of Comput...
«Innovations in pedagogy using MOOCs» /  Ting-Chuen Pong, professor of Comput...«Innovations in pedagogy using MOOCs» /  Ting-Chuen Pong, professor of Comput...
«Innovations in pedagogy using MOOCs» / Ting-Chuen Pong, professor of Comput...
 
Making MOOCs and changing open educational practices
Making MOOCs and changing open educational practicesMaking MOOCs and changing open educational practices
Making MOOCs and changing open educational practices
 
From OpenCourseWare to Open CourseWare
From OpenCourseWare to Open CourseWareFrom OpenCourseWare to Open CourseWare
From OpenCourseWare to Open CourseWare
 
How to MOOC? – A pedagogical guideline for practitioners
How to MOOC? – A pedagogical guideline for practitionersHow to MOOC? – A pedagogical guideline for practitioners
How to MOOC? – A pedagogical guideline for practitioners
 
Online learning for MOOC team developers
Online learning for MOOC team developersOnline learning for MOOC team developers
Online learning for MOOC team developers
 
Learning design and learning analytics
Learning design and learning analyticsLearning design and learning analytics
Learning design and learning analytics
 
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
EMMA Summer School - Rebecca Ferguson - Learning design and learning analytic...
 
Df e rr355_-_opportunities_for_moo_cs_in_schools_final
Df e rr355_-_opportunities_for_moo_cs_in_schools_finalDf e rr355_-_opportunities_for_moo_cs_in_schools_final
Df e rr355_-_opportunities_for_moo_cs_in_schools_final
 
Supporting educators as designers of complex blended learning scenarios: visu...
Supporting educators as designers of complex blended learning scenarios: visu...Supporting educators as designers of complex blended learning scenarios: visu...
Supporting educators as designers of complex blended learning scenarios: visu...
 
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth CobosMeta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
Meta-review of recognition of learning in LMS and MOOCs - Ruth Cobos
 
Learning Analytics: Realizing their Promise in the California State University
Learning Analytics:  Realizing their Promise in the California State UniversityLearning Analytics:  Realizing their Promise in the California State University
Learning Analytics: Realizing their Promise in the California State University
 
Learning Analytics for MOOCs: EMMA case
Learning Analytics for MOOCs: EMMA caseLearning Analytics for MOOCs: EMMA case
Learning Analytics for MOOCs: EMMA case
 
Moocs
MoocsMoocs
Moocs
 
Some Issues Affecting the Sustainability of Open Learning Courses
Some Issues Affecting the Sustainability of Open Learning Courses Some Issues Affecting the Sustainability of Open Learning Courses
Some Issues Affecting the Sustainability of Open Learning Courses
 
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
VII Jornadas eMadrid "Education in exponential times". Mesa redonda eMadrid L...
 
Qmm sept 22 2015
Qmm sept 22 2015Qmm sept 22 2015
Qmm sept 22 2015
 
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
2018-07-10 ICALT 2018 in Mumbai Gap between MOOC Designers and Learners on In...
 

More from Vitomir Kovanovic

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
Vitomir Kovanovic
 
Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...
Vitomir Kovanovic
 
Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...
Vitomir Kovanovic
 
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Vitomir Kovanovic
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
Vitomir Kovanovic
 
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Vitomir Kovanovic
 
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Vitomir Kovanovic
 
Unsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics ResearchersUnsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics Researchers
Vitomir Kovanovic
 
Introduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics ResearchersIntroduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics Researchers
Vitomir Kovanovic
 
A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...
Vitomir Kovanovic
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
Vitomir Kovanovic
 
Introduction to Epistemic Network Analysis
Introduction to Epistemic Network AnalysisIntroduction to Epistemic Network Analysis
Introduction to Epistemic Network Analysis
Vitomir Kovanovic
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
Vitomir Kovanovic
 
Assessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methodsAssessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methods
Vitomir Kovanovic
 
What is the source of social capital? The association between social network ...
What is the source of social capital? The association between social network ...What is the source of social capital? The association between social network ...
What is the source of social capital? The association between social network ...
Vitomir Kovanovic
 
Automated Cognitive Presence Detection in Online Discussion Transcripts
Automated Cognitive Presence Detection in Online Discussion TranscriptsAutomated Cognitive Presence Detection in Online Discussion Transcripts
Automated Cognitive Presence Detection in Online Discussion Transcripts
Vitomir Kovanovic
 

More from Vitomir Kovanovic (16)

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
 
Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...
 
Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...
 
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
 
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
 
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...
 
Unsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics ResearchersUnsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics Researchers
 
Introduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics ResearchersIntroduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics Researchers
 
A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
 
Introduction to Epistemic Network Analysis
Introduction to Epistemic Network AnalysisIntroduction to Epistemic Network Analysis
Introduction to Epistemic Network Analysis
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
 
Assessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methodsAssessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methods
 
What is the source of social capital? The association between social network ...
What is the source of social capital? The association between social network ...What is the source of social capital? The association between social network ...
What is the source of social capital? The association between social network ...
 
Automated Cognitive Presence Detection in Online Discussion Transcripts
Automated Cognitive Presence Detection in Online Discussion TranscriptsAutomated Cognitive Presence Detection in Online Discussion Transcripts
Automated Cognitive Presence Detection in Online Discussion Transcripts
 

Recently uploaded

Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
danielkiash986
 
Electric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger HuntElectric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger Hunt
RamseyBerglund
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
ssuser13ffe4
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
PsychoTech Services
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
RidwanHassanYusuf
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
National Information Standards Organization (NISO)
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
zuzanka
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skillsspot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
haiqairshad
 
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching AptitudeUGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
S. Raj Kumar
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
deepaannamalai16
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Nguyen Thanh Tu Collection
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Henry Hollis
 
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
imrankhan141184
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Denish Jangid
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
siemaillard
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
RAHUL
 

Recently uploaded (20)

Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
Pharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brubPharmaceutics Pharmaceuticals best of brub
Pharmaceutics Pharmaceuticals best of brub
 
Electric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger HuntElectric Fetus - Record Store Scavenger Hunt
Electric Fetus - Record Store Scavenger Hunt
 
math operations ued in python and all used
math operations ued in python and all usedmath operations ued in python and all used
math operations ued in python and all used
 
Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
 
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptxBIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
BIOLOGY NATIONAL EXAMINATION COUNCIL (NECO) 2024 PRACTICAL MANUAL.pptx
 
Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"Benner "Expanding Pathways to Publishing Careers"
Benner "Expanding Pathways to Publishing Careers"
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skillsspot a liar (Haiqa 146).pptx Technical writhing and presentation skills
spot a liar (Haiqa 146).pptx Technical writhing and presentation skills
 
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching AptitudeUGC NET Exam Paper 1- Unit 1:Teaching Aptitude
UGC NET Exam Paper 1- Unit 1:Teaching Aptitude
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
 
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.pptLevel 3 NCEA - NZ: A  Nation In the Making 1872 - 1900 SML.ppt
Level 3 NCEA - NZ: A Nation In the Making 1872 - 1900 SML.ppt
 
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
Traditional Musical Instruments of Arunachal Pradesh and Uttar Pradesh - RAYH...
 
Chapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptxChapter wise All Notes of First year Basic Civil Engineering.pptx
Chapter wise All Notes of First year Basic Civil Engineering.pptx
 
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptxPrésentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
Présentationvvvvvvvvvvvvvvvvvvvvvvvvvvvv2.pptx
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
 

Kovanović et al. 2017 - developing a mooc experimentation platform: insights from a user study

  • 1. DEVELOPING A MOOC EXPERIMENTATION PLATFORM: INSIGHTS FROM A USER STUDY The University of Edinburgh Vitomir Kovanović, Srećko Joksimović, Philip Katerinopoulos, Charalampos Michail, George Siemens, & Dragan Gašević Simon Fraser University, Vancouver, Canada Lak17 @vkovanovic v.kovanovic@ed.ac.uk http://vitomir.kovanovic.info
  • 2. WHY ARE MOOCS INTERESTING (FOR A RESEARCHER) MOOCs 2 Lak17Kovanovic et al. – Developing a MOOC experimentation platform 1. Educational researchers historically small scale – “Big Data” in education 2. Potential for improving on-campus learning experience 3. Innovate for teaching innovations (e.g., flipped classroom models) 4. Great potential for understanding human learning 5. Experiment with different instructional interventions
  • 3. WHY ARE MOOCS INTERESTING (FOR A RESEARCHER) MOOCs 3 Lak17Kovanovic et al. – Developing a MOOC experimentation platform 1. Educational researchers historically small scale – “Big Data” in education 2. Potential for improving on-campus learning experience 3. Innovate for teaching innovations (e.g., flipped classroom models) 4. Great potential for understanding human learning 5. Experiment with different instructional interventions ALSO (of course) 1. Democratization of learning 2. Helping students without access to good quality education 3. Closing the achievement gap 4. Helping developing countries 5. Reduce costs of education 6. …..
  • 4. CURRENT RESEARCH CHALLENGES MOOCs 4 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 5. CURRENT RESEARCH CHALLENGES MOOCs 5 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 6. CURRENT RESEARCH CHALLENGES MOOCs 6 Lak17Kovanovic et al. – Developing a MOOC experimentation platform Data is notoriously had to collect and analyze 1. Diverse data sources 2. Tracking users across platforms (e.g., twitter, surveys) 3. Different data formats (CSV formats, JSON formats) 4. Disconnected data sources (_general, _mappings, _forum) 5. Anonymization 6. IRBs 7. Data analysis requires understanding of the data science methods (e.g., feature engineering) 8. Understanding of the pedagogical domain 9. Correctly interpreting the data 10. Issues with replication of educational studies 11. No ad-hoc experimentation, A/B testing must be planned far in advance
  • 7. CURRENT RESEARCH CHALLENGES MOOCs 7 Lak17Kovanovic et al. – Developing a MOOC experimentation platform 1. Most MOOC instructors lack these skills 1. One reason for why Computer Science researches dominated MOOC research
  • 8. GOAL MOOCs 8 Lak17Kovanovic et al. – Developing a MOOC experimentation platform Develop a tool that will assist MOOC instructors/researchers in analyzing the MOOC data Guiding principles: 1. No technical prerequisites 1. No database installs 2. Incremental data import 1. As the new data dump becomes available, it should be possible to import it. 3. Support cluster analysis 1. We will add additional types of analysis over time (e.g., SNA) 4. Enable follow-up analysis 1. Details about interventions should be possible to analyze in R/SPSS 5. Support class intervention 1. Instructors should be able to send emails to selected groups of students 6. Support study replication 1. Support pre-existing analysis within the tool
  • 9. GOAL MOOCs 9 Lak17Kovanovic et al. – Developing a MOOC experimentation platform Develop a tool that will assist MOOC instructors/researchers in analyzing the MOOC data Design decisions: 1. For now, desktop based 1. Later maybe provide a web-based version 2. Data sharing & IRB issues 2. No fancy database scheme abstractions, focus on one platform at a time 1. Coursera session-based implemented, edX in progress 3. Java-based to support multiple platforms
  • 10. OVERVIEW TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 10
  • 11. OVERVIEW TAB MOOCito 11 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 12. DETAILS TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 12
  • 13. ANALYZE TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 13
  • 14. ANALYZE | INDICATORS TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 14
  • 15. ANALYZE | RUN ANALYSIS TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 15
  • 16. ANALYZE | EVALUATE TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 16
  • 17. ANALYZE | EVALUATE TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 17
  • 18. INTERVENE TAB MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 18
  • 19. EVALUATION MOOCito Lak17Kovanovic et al. – Developing a MOOC experimentation platform 19 1. Conducted a user study with 11 participants (UK, USA, South America) 1. All participants were experienced with online learning, MOOCs, and Coursera platform 2. Case scenario of a hypothetical MOOC instructor halfway through the course 3. Survey questionnaire based on technology acceptance model (TAM)
  • 20. STUDY MOOCito 20 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 21. RESULTS MOOCito 1. Overall, participants were satisfied with the proposed system and its perceived usability and ease-of-use. 2. Most positive responses: 1. The ability to generate interesting questions and hypotheses (4.73), 2. The ability of the tool to provide insights into students’ engagement with the learning environment (4.64), and 3. The capacity to provide different interventions to subgroups of students (4.55). 3. In open-ended responses, participants emphasized: 1. The selection of engagement indicators, weekly plots, and clear interface. 2. The flexibility and ease of performing interventions without “the dangers of providing blanket information inappropriately” to a particular target subset of students 3. The use of templates and “variables” in the message body. 4. Strong willingness (4.36) to use MOOCito and would like to be able to use it in their courses (4.00). 1. Willingness to use it for research was higher (4.55) than for course design and teaching purposes (4.30). 21 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 22. ISSUES MOOCito 1. Concerns regarding the accessibility of the tool for “typical” MOOC instructors. 1. Not familiar with: 1. Terminology, 2. Engagement indicators, 3. Statistical analyses, or 4. Study “templates” 2. Future work: 1. Short summaries, 2. Video tutorials and 3. Bibliographical information 2. What would be the best way to analyze the data? 1. Future work: Additional information on the “standard approaches” 3. Simplification of the analysis steps (D4) 1. Future work: More streamlined analysis, better guidelines and support for instructors with little background and experience in statistics. 4. Instructors should be able to go from indicators of engagement to the actual student content. 5. Challenge of knowing when or how to intervene, 1. The need for more “interpretable” results, summarized in a form which is easy to act upon. 2. Simple descriptions using “low,” “moderate,” and “high” for describing engagement 22 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 23. PROGRESS MOOCito 1. Support for edX and Coursera rolling session format 2. Work on user interface 3. Work on SNA analysis 4. Modularizing it so it is extensible 5. “Interpretable results” 23 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 24. SUMMARY MOOCito “This has the potential to be a powerful tool in educational research as well as a means of tracking learner engagement day-to-day.”. 24 Lak17Kovanovic et al. – Developing a MOOC experimentation platform
  • 25. THANK YOU QUESTIONS? Web: http://vitomir.kovanovic.info Mail: v.kovanovic@ed.ac.uk Twitter: @vkovanovic LUNCH IS GETTING COLD The end 25 Lak17Kovanovic et al. – Developing a MOOC experimentation platform