Presentation at The Institute for Adaptive and Neural Computation (ANC), The University of Edinburgh workshop series

1. MOOCs & Social Learning
Challenges and opportunities
Vitomir Kovanovic
School of Informatics
The University of Edinburgh
v.kovanovic@ed.ac.uk
ANC Workshop
2 June 2015

2. MOOCs: an overview
● Major hype in EdTech world since 2011
● Millions of $$$ raised
● 2012 “year of the MOOCs”
● Led by highly respected institutions
● Attracted interest of general public
○ “Tsunami in world of education”
○ “Disruptive change”
● Primarily content-focused
○ short video lectures
○ quizzes
2

3. MOOCs: early starts
● MOOCs were envisioned as a platform for connectivist learning
● First MOOC: 2008 Connectivism and Connective Knowledge (CCK08)
by G. Siemens & S. Downes
● Little resemblance with today’s “Coursera-style” MOOCs
○ Novel “post-industrialist” form of learning (Anderson & Dron, 2010)
○ Learning is about building connections (with content, people, and organizations)
○ Self-directed, no “formal” assessment, no certificates
○ Focus on building communities and starting up interesting conversations
○ Student interests define topics, instructors are there to support
○ Every week a new lecture on a particular topic
○ Students write blogs, research the domain, start conversation in their own social media space
■ focus on self-reflection and social interactions
○ Every week there is an aggregation email with links to all produced content
3

4. cMOOCs vs. xMOOCs
● Behaviorist MOOCs (xMOOCs) and Connectivist MOOCs (cMOOCs)
xMOOCs cMOOCs
Scalability of provision Massive Community and connections
Open access - Restricted license Open Open access & license
Individual learning in a single
platform
Online Networked learning across multiple
platforms and services
Acquire a curriculum of knowledge &
skills
Course Develop shared practices, knowledge and
understanding
Comparison of xMOOCs and cMOOCs by Yuan, Powell, & Olivier (2014)
4

5. cMOOCs vs. xMOOCs
5

6. MOOCs: Revolution or evolution?
Three generations of distance education pedagogies (Anderson & Dron, 2010):
Cognitive-Behaviorism -> Social Constructivism -> Connectivism
Modern educational psychology:
● Learners do not acquire knowledge, they construct knowledge
● Learners are agents making decisions about their own learning tactics and approaches
MOOCS are a current stage of progress in evolution of distance education
● MOOCs were envisioned as social-constructivism 2.0
● In some aspects, xMOOCs are even a step back in online learning
○ Step back to cognitive-behaviorist learning models
○ xMOOCs were a step back because of practical reasons
● We need to look what we already know from distance/online learning (Kovanović et al., 2015)
6

7. Trends & challenges
● How we can use MOOCS for improving face-to-face courses and traditional online courses?
○ Move toward digital learning: a blend of face-to-face learning, online learning and MOOCs
(Siemens, Gašević, & Dawson, 2015)
● How to make MOOC experience more social?
○ Dual models (c+xMOOCs) (Dawson, Joksimović, Kovanović, Gašević, & Siemens, 2015)
● How assessment should look like?
○ What it means to “complete” the course? How about not having courses at all?
● Media coverage is rapidly decreasing (Kovanović, Joksimović, Gašević, Siemens, & Hatala, 2014)
○ MOOCs are not new anymore
○ Topics with a growing interest:
■ government regulations
■ adoption in different parts of the world
■ use of data & analytics
7

8. Goal: More social MOOCs
● Goal to enable for an environment in which students are able to learn together at scale
○ Online discussions should be better
■ Currently work mostly as Q/A
■ More knowledge building in discussions
○ Currently, students are having solitary experiences in MOOCs at scale
● Build on the existing models of online learning
○ Community of Inquiry model (Garrison, Anderson, & Archer, 1999)
● Use Learning Analytics and Data Mining to achieve research goal
○ Discourse analysis and social network analysis
○ Build tools that can be used by instructors
○ Extend knowledge about social learning processes
8

9. Community of Inquiry (CoI) model
Social
presence
Cognitive
presence
Teaching
presence
Educational
experience
1. Affectivity
2. Interactivity
3. Group cohesion
1. Triggering event: Problem
identification, sense of
puzzlement
2. Exploration: Brainstorming,
Idea exploration, divergence
3. Integration: Synthesis of
relevant information
4. Resolution: Problem resolution,
testing application
1. Design & Organization
2. Direct instruction
3. Facilitation
Garrison, Anderson, and Archer (1999) 9

10. CoI instruments
Quantitative coding schemes for each of the presences:
● Labour-intensive manual coding
● Requires experienced coders
34 likert items survey instrument
● 13 Teaching presence
● 9 Social presence
● 12 Cognitive presence
10

11. MOOCs: Challenges
CoI (and other social-constructivist) models require a strong teacher’s presence
-> up to 30-40 student cohorts (Anderson & Dron, 2010)
MOOCs?
● In short, just too many students for strong teaching presence during course
11

12. How different is MOOC context?
● We evaluated CoI survey
instrument
● EFA of existing CoI survey
instrument using data from 5
MOOCs
● Course design & organization
are of particular importance
● Less affective communication
● Less resolution
Resolution &
application
Affectivity
Course design
& organization
12

13. How different is MOOC context?
● SEM model of relationships between
presences
● Main findings:
○ Strong direct effect of teaching
presence on cognitive presence
○ Social presence acts as a mediator
between teaching and cognitive
presences
13

14. How different is MOOC context?
● SEM model of relationships between
presences
● Still in progress
○ Moderately good fit (RMSEA = 0.09)
○ Stronger direct effect of teaching on
cognitive presence
○ Lesser mediating effect of social
presence
14

15. CoI content analysis
● Besides survey, CoI coding scheme for each of the presences
● For each of the presences, quantitative coding scheme
● Message unit of analysis
15

16. CoI content analysis
16

17. Challenges of content analysis
● Very labor intensive
● Crude coding scheme
● Requires experienced coders
● Can’t be used for real-time monitoring
● Not explaining reasons behind observed levels of presences
● Used for analysis of learning long after courses are over
17

18. Research approach
● Use text analytics to address these problems.
● Develop automated content analysis system for message coding.
● Provide better operationalization of the CoI coding instrument.
18

19. Dataset
● Six offerings of graduate level course in
software engineering.
● Total of 1747 messages, 81 students.
● Manually coded by two coders (agreement =
98.1%, Cohen’s κ = 0.974).
● Currently coding E-learning and Digital
Cultures UoE MOOC
ID Phase No. Messages (%)
0 Other 140 (8.01%)
1 Triggering Event 308 (17.63%)
2 Exploration 684 (39.17%)
3 Integration 508 (29.08%)
4 Resolution 107 (6.12%)
All phases 1747 (100%)
19

20. Methodology
● SVM classifier with RBF kernel.
● Parameter tuning & accuracy evaluated using nested 5-fold cross-validation.
● Extracted features:
○ N-grams
○ Part-of-Speech N-grams
○ Back-Off N-grams
○ Dependency Triplets
○ Back-Off Dependency Triplets
○ Named Entities
○ Thread Position Features
○ LSA Features
○ LIWC Features
20

21. Results
● We achieved Cohen’s κ of 0.42 for our classification problem
● Better than the existing Neural Network system (Cohen’s κ=0.31)
● Unigram baseline model achieved Cohen’s κ of 0.33
21
(Kovanović, Joksimović, Gašević, & Hatala, 2014)

22. Challenges
● Disproportionate class sizes
● Effect of the code-up rule for coding
● Context is not taken into the account
● No explanatory value
Code-up coding rule
22

23. In progress
• Discussions (and
students’ learning)
progresses from triggering
to resolutions.
• Content of a message
depends on the content of
the previous messages.
• Content of a message
depends on the learning
progress of a given
student.
23

24. Summary
● Social-constructivist pedagogies do not work well in MOOC context
○ xMOOCs: focus on content-delivery
○ cMOOCs: focus on connecting, reflecting in a self-directed way
● MOOCs
○ put more emphasis on course organization and design
○ building community a challenge
○ focus more on remembering than on understanding and evaluating
○ need for a more social experience in MOOCs
● Community of Inquiry model
○ Widely used, hard to adopt
○ Automated coding of messages:
● An overview of student progress in development of cognitive presence
● Support instructional interventions
● Further understanding of CoI model, particularly in MOOC context
● More detailed operationalization of CoI coding scheme
24

25. 25
Thank you

26. References
Anderson, T., & Dron, J. (2010). Three generations of distance education pedagogy. The International Review of Research in
Open and Distance Learning, 12(3), 80–97.
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical Inquiry in a Text-Based Environment: Computer Conferencing in
Higher Education. The Internet and Higher Education, 2(2–3), 87–105.
Kovanović, V., Joksimović, S., Gašević, D., Siemens, G., & Hatala, M. (2014). What public media reveals about MOOCs?
British Journal of Educational Technology 43(3), 510-527.
Dawson, S., Joksimović, S., Kovanović, V., Gašević, D., & Siemens, G. (2015). Recognising learner autonomy: Lessons and
reflections from a joint x/c MOOC. In Proceedings of 2015 HERDSA conference. Melbourne, AU.
Siemens, G., Gašević, D., & Dawson, S. (2015). Preparing for the digital university: a review of the history and current state of
distance, blended, and online learning. Athabasca University. Retrieved from http://linkresearchlab.
org/PreparingDigitalUniversity.pdf
Kovanović, V., Joksimović, S., Skrypnyk, O., Gašević, D., Dawson, S., & Siemens, G. (2015). The history and state of distance
education. Athabasca University.
Kovanović, V., Joksimović, S., Gašević, D., & Hatala, M. (2014). Automated Content Analysis of Online Discussion Transcripts.
In Proceedings of the Workshops at the LAK 2014 Conference co-located with 4th International Conference on Learning
Analytics and Knowledge (LAK 2014). Indianapolis, IN. Retrieved from http://ceur-ws.org/Vol-1137/
Yuan, L., Powell, S., & Olivier, B. (2014). Beyond MOOCs: Sustainable Online Learning in Institutions. CETIS: Centre for
Educational Technology, Interoperability and Standards.
26