SlideShare a Scribd company logo
1 of 38
Download to read offline
Learning Analytics for Communities of Inquiry
Vitomir Kovanovi´c1,3
Dragan Gaˇsevi´c1,2
Marek Hatala3
v.kovanovic@ed.ac.uk dgasevic@acm.org mhatala@sfu.ca
1
School of Informatics 2
Moray House School of Education
University of Edinburgh University of Edinburgh
Edinburgh, United Kingdom Edinburgh, United Kingdom
3
School of Interactive Arts and Technology
Simon Fraser University
Burnaby, BC, Canada
March 17, 2015
Marist College,
Poughkeepsie, NY, USA
Introduction
Ph.D research Overview
High-level
The goal of my PhD research is to expand the
knowldge about inquiry-based digital learning
through the development of novel learning
analytics models.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 1 / 22
Introduction
Ph.D research Overview
High-level
The goal of my PhD research is to expand the
knowldge about inquiry-based digital learning
through the development of novel learning
analytics models.
Low-level
I’m looking at how use of text analytics, social
network analysis and trace data clustering can
be used to improve the understanding of the
Community of Inquiry (CoI) model of distance
education.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 1 / 22
Background Inquiry-based online learning
Asynchronous online discussions
• Primary means of social interaction in online
communities,
• Frequently used for various types of education
delivery (e.g., blended, F2F, distance),
• Well supported by the social-constructivist
pedagogies,
• Can serve multiple purposes (e.g., social support,
community building, QA, student-instructor
interaction, social learning),
• Widely used, often not appropriately,
• Require substantial effort from instructors.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 2 / 22
Background Community of Inquiry (CoI) Framework
Community of Inquiry (CoI) framework
Community of Inquiry is a conceptual framework outlying the important constructs
that define worthwhile educational experience in distance education setting [3].
• Social presence: relationships and social
climate in a community.
• Cognitive presence: phases of cognitive
engagement and knowledge construction.
• Teaching presence: instructional role
during social learning.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 3 / 22
Background Community of Inquiry (CoI) Framework
Community of Inquiry (CoI) framework
Community of Inquiry is a conceptual framework outlying the important constructs
that define worthwhile educational experience in distance education setting [3].
CoI framework is:
• Extensively researched and validated,
• Widely used in distance education
research.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 3 / 22
Background Community of Inquiry (CoI) Framework
Cognitive Presence
Cognitive Presence
“an extent to which the participants in any particular configuration of a
community of inquiry are able to construct meaning through sustained
communication.” [3, p .89]
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 4 / 22
Background Community of Inquiry (CoI) Framework
Cognitive Presence
Cognitive Presence
“an extent to which the participants in any particular configuration of a
community of inquiry are able to construct meaning through sustained
communication.” [3, p .89]
Four phases of cognitive presence:
1 Triggering event: Some issue, dilemma or problem is identified.
2 Exploration: Students move between private world of reflection and shared
world of social knowledge construction.
3 Integration: Students filter irrelevant information and synthesize new
knowledge.
4 Resolution: Students analyze practical applicability, test different
hypotheses, and start a new learning cycle.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 4 / 22
Background Community of Inquiry (CoI) Framework
CoI coding instrument
• Based on quantitative content analysis (QCA) [11],
• Defines three coding schemes for each of the presences,
• Use of whole message as unit of analysis,
• Look for particular indicators of different sociocognitive processes,
• Additional heuristics,
• Code-up: When a message clearly displays indicators of several phases it is
coded to the latest phase, and
• Code-down: When it is not clear which phase is reflected, code to the earliest
phase.
• Requires expertise with coding instrument and domain knowledge.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 5 / 22
Background Community of Inquiry (CoI) Framework
CoI coding instrument
• Based on quantitative content analysis (QCA) [11],
• Defines three coding schemes for each of the presences,
• Use of whole message as unit of analysis,
• Look for particular indicators of different sociocognitive processes,
• Additional heuristics,
• Code-up: When a message clearly displays indicators of several phases it is
coded to the latest phase, and
• Code-down: When it is not clear which phase is reflected, code to the earliest
phase.
• Requires expertise with coding instrument and domain knowledge.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 5 / 22
Background Community of Inquiry (CoI) Framework
CoI challenges
• Practical challenges, related to its adoption by instructors,
• Theoretical challenges, related to current understanding of social
interactions, human agency and technology use.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 6 / 22
Background Community of Inquiry (CoI) Framework
CoI practical challenges
• Content analysis instrument:
• Time consuming, labor intensive manual message coding,
• Hard to scale, typically used for small sample studies,
• Typically used for research purposes after courses are over.
As a result, content analysis (including CoI) had almost no impact on
educational practice [2].
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
Background Community of Inquiry (CoI) Framework
CoI practical challenges
• Content analysis instrument:
• Time consuming, labor intensive manual message coding,
• Hard to scale, typically used for small sample studies,
• Typically used for research purposes after courses are over.
As a result, content analysis (including CoI) had almost no impact on
educational practice [2].
• Survey instrument:
• Removes the need for manual coding,
• Administered at the end of the course,
• Does not allow any feedback for the duration of the course.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
Background Community of Inquiry (CoI) Framework
CoI practical challenges
• Content analysis instrument:
• Time consuming, labor intensive manual message coding,
• Hard to scale, typically used for small sample studies,
• Typically used for research purposes after courses are over.
As a result, content analysis (including CoI) had almost no impact on
educational practice [2].
• Survey instrument:
• Removes the need for manual coding,
• Administered at the end of the course,
• Does not allow any feedback for the duration of the course.
• A need for more proactive use of available data,
• Explanations for observed levels of thee presences,
• Suggestions and guidelines for instructors to direct their instructional
interventions.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
Background Community of Inquiry (CoI) Framework
CoI theoretical challenges
Although CoI framework is rooted in social constructivism, there is barely any
mention of social networks and social capital in current CoI literature:
• How does student position in social network reflect on the development of
social and cognitive presence and ultimately on learning outcomes?
• What types of interactions promote the development of students’ social
capital within communities of inquiry?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
Background Community of Inquiry (CoI) Framework
CoI theoretical challenges
Although CoI framework is rooted in social constructivism, there is barely any
mention of social networks and social capital in current CoI literature:
• How does student position in social network reflect on the development of
social and cognitive presence and ultimately on learning outcomes?
• What types of interactions promote the development of students’ social
capital within communities of inquiry?
Human agency and self-regulation of learning are also not frequently discussed:
• How does students use available technology to learn in inquiry-based courses?
• How does different technology use affects development of cognitive presence,
attainment of learning objectives and course performance?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
Background Community of Inquiry (CoI) Framework
CoI theoretical challenges
Although CoI framework is rooted in social constructivism, there is barely any
mention of social networks and social capital in current CoI literature:
• How does student position in social network reflect on the development of
social and cognitive presence and ultimately on learning outcomes?
• What types of interactions promote the development of students’ social
capital within communities of inquiry?
Human agency and self-regulation of learning are also not frequently discussed:
• How does students use available technology to learn in inquiry-based courses?
• How does different technology use affects development of cognitive presence,
attainment of learning objectives and course performance?
CoI adoption outside typical small-class DE courses is not widely described:
• What are the challenges of CoI use in larger courses (e.g., MOOCs)?
• Under what conditions can CoI be used in large online courses?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
Research questions
Research questions
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 9 / 22
Research questions
Research questions
• How to enable for easier and more scalable quantitative content analysis in
accordance with community of inquiry coding schemes?
• Which social processes, and to what extent, are indicative of the development
of the social capital in communities of inquiry?
• What is the relationship between students’ social capital and the
development of cognitive presence?
• With respect to the development of cognitive presence, are central positions
within student social networks beneficial or not?
• What are the main technology use profiles withing communities of inquiry?
• How does different technology-use profile affect the development of cognitive
presence and attainment of learning objectives?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 9 / 22
Proposed approach
Proposed approach
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
Proposed approach
Proposed approach
Proposed approach
Development of learning analytics for communities of inquiry.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
Proposed approach
Proposed approach
Proposed approach
Development of learning analytics for communities of inquiry.
In general,
Build supervised and unsupervised analytical models to address the
existing challenges with Community of Inquiry framework.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
Proposed approach
Proposed approach
Proposed approach
Development of learning analytics for communities of inquiry.
More precisely,
1 Develop a text analytics system for automated message coding in
accordance with cognitive presence coding scheme,
2 Develop a predictive model for understanding the relationship
between social network capital and social presence,
3 Identify technology-use profiles based on trace data clustering and
examine their relationship with development of cognitive presence
and learning outcomes.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
Proposed approach
Text analytics system
• Development of a novel text classifier for CoI message coding,
• There have been few attempts to automate CoI coding process [12, 1],
• Their accuracy not sufficient for practical adoption,
• Based on popular classification techniques and surface text features.
• Warrants development of novel text classification method that is more
suitable for a given problem,
• Provides a more detailed operationalization of cognitive presence coding
scheme.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 11 / 22
Proposed approach
Text analytics system: typical approach
• Pose a problem as a traditional multi-class supervised classification problem,
• Use of surface-level features (N-grams),
• Use of part-of-speech (POS) tags,
• Use of combinations of N-grams and POS tags (back-off N-grams),
• Use of grammatical dependency triplets,
• Thread position features,
• Standard classification algorithms (SVM, Naive Bayes),
• Cross validation a preferred method of validation accuracy and parameters
optimization,
• In order to be done correctly, requires nesting of cross-validation.
• Does not capture code-up and code-down rules,
• Issues with quoting of messages, and
• Issues with rare classes (i.e., resolution phase).
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 12 / 22
Proposed approach
Text analytics system: typical approach
• Pose a problem as a traditional multi-class supervised classification problem,
• Use of surface-level features (N-grams),
• Use of part-of-speech (POS) tags,
• Use of combinations of N-grams and POS tags (back-off N-grams),
• Use of grammatical dependency triplets,
• Thread position features,
• Standard classification algorithms (SVM, Naive Bayes),
• Cross validation a preferred method of validation accuracy and parameters
optimization,
• In order to be done correctly, requires nesting of cross-validation.
• Does not capture code-up and code-down rules,
• Issues with quoting of messages, and
• Issues with rare classes (i.e., resolution phase).
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 13 / 22
Proposed approach
Text analytics system: better approach
• Take into the account the class (i.e., phase) of the previous message,
• Take into the account the phase of cognitive development of a given student,
and
• Take into the account code-up and code-down rules,
• Code-up needs a feature overriding,
• Code-down needs an estimate of classification certainty,
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 14 / 22
Proposed approach
Challenges
• What features of discourse are good indicators that can be used for building
a classification algorithm?
• How to support and leverage the cyclic nature of cognitive presence?
• How to model feature precedence?
• What features are indicative of the different phases of cognitive presence?
• How to build automated text classification system that has accuracy high
enough to warrant its practical use?
• How to automate as much of message coding as possible while preserving
sufficiently high classification accuracy?
• How to develop a system that is not a domain dependent?
• How to speed up train of the system in new domain?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 15 / 22
Proposed approach
Social analytics system
• Development of learning analytics based on social network analysis of student
interactions,
• Understand the role of social presence on the development of social capital,
• Understand the link between social network position and the development of
cognitive presence.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 16 / 22
Proposed approach
Technology-use profiles
• Development of learning analytics based on student trace-data clustering,
• Understand the different ways of educational technology-use,
• Understand the link between different technology-use profiles and
development of cognitive presence,
• Understand the link between different technology-use profiles and success in
the course.
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 17 / 22
Proposed approach
Challenges
• How to develop a system that is not a domain dependent?
• How to speed up train of the system in new domain?
• How to identify technology-use profiles?
• What forms of social presence are indicative of student social capital?
• What trace data to use for profile identification?
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 18 / 22
Methodology
Methodology
1 Literature review
• Educational research
• Content analysis in distance
education
• Community of Inquiry
• Social network analysis
• Data mining
• Text mining
• Online discussion and newsgroup
classification
• Clustering
• Graph-based data mining
• Quantitative research methods
2 Data collection
• Selected Topics in Software
Engineering course
• UoE E-learning and digital cultures
MOOC
3 Development of classification algorithm
• Feature extraction
• Algorithm development
• Validation
4 Technology-use profiling
• Trace data conversion and feature
extraction
• Cluster discovery and validation
• Examination of relationships with
cognitive presence and learning
outcomes
5 Social network analysis
• Social graph extraction
• Longitudinal analysis of relationships
between network centrality and
learning outcomes
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 19 / 22
Current Progress
Current progress
1 Literature review
• Educational research
• Content analysis in distance
education
• Community of Inquiry
• Social network analysis
• Data mining
• Text mining
• Online discussion and newsgroup
classification
• Clustering
• Graph-based data mining
• Quantitative research methods
2 Data collection
• Selected Topics in Software
Engineering course
• UoE E-learning and digital cultures
MOOC
3 Development of classification algorithm
• Feature extraction
• Algorithm development
• Validation
4 Technology-use profiling
• Trace data conversion and feature
extraction
• Cluster discovery and validation
• Examination of relationships with
cognitive presence and learning
outcomes
5 Social network analysis
• Social graph extraction
• Longitudinal analysis of relationships
between network centrality and
learning outcomes
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 20 / 22
Current Progress
Publications
• Preliminary classifier implementation [6]
• End of course CoI SNA analysis [7]
• MOOC Research analysis [4]
• MOOC News analysis [10]
• Time-on-task estimation challenges [9]
• Analysis of cognitive presence linguistic properties [5]
• JLA doctoral proposal paper [8]
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 21 / 22
Expected contributions
Benefits of the proposed research:
• Based on the empirical evidence, expand current state of understanding of
distance education by investigating learners’ interactions with information
(i.e., content), technology, instructors and other learners.
• Make a foundation on which improvements on the current educational
research and practice could be developed.
• Provide mode detailed operationalization of the CoI coding instrument by the
means of text analytics,
• Provide basis for easier adoption of CoI model by the researchers and
practitioners,
Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 22 / 22
Thank you
References I
Stephen Corich, Kinshuk Hunt, and Lynn Hunt. “Computerised Content Analysis for Measuring Critical
Thinking within Discussion Forums”. In: Journal of e-Learning and Knowledge Society 2.1 (2012).
Roisin Donnelly and John Gardner. “Content analysis of computer conferencing transcripts”. In:
Interactive Learning Environments 19.4 (2011), pp. 303–315.
D. Randy Garrison, Terry Anderson, and Walter Archer. “Critical Inquiry in a Text-Based Environment:
Computer Conferencing in Higher Education”. In: The Internet and Higher Education 2.2–3 (1999),
pp. 87–105.
Dragan Gasevic et al. “Where is Research on Massive Open Online Courses Headed? - A data analysis
of the MOOC Research Initiative”. In: The International Review of Research in Open and Distance
Learning submitted (2014).
Sre´cko Joksimovi´c et al. “Psychological characteristics in cognitive presence of communities of inquiry:
A linguistic analysis of online discussions”. In: The Internet and Higher Education 22 (2014), pp. 1–10.
Vitomir Kovanovic et al. “Automated Content Analysis of Online Discussion Transcripts”. In:
Proceedings of the 2014 Learning Analytics and Knowledge (LAK) Conference Workshop on Machine
Learning and Learning Analytics. 2014.
References II
Vitomir Kovanovic et al. “What is the source of social capital? The association between social network
position and social presence in communities of inquiry”. In: The first International Workshop on
Graph-based Educational Datamining (G-EDM) 2014. 2014.
Vitomir Kovanovi´c and Dragan Gaˇsevi´c. “Learning Analytics for Communities of Inquiry”. In: Journal
of Learning Analytics in-press (2014).
Vitomir Kovanovi´c et al. “Penetrating the Black Box of Time-on-task Estimation”. In: Proceedings of
the Fifth International Conference on Learning Analytics and Knowledge (LAK 2015). Best paper
award nominee. 2015.
Vitomir Kovanovi´c et al. “What public media reveals about MOOCs?” In: British Journal of
Educational Technology accepted (2014).
Klaus H. Krippendorff. Content Analysis: An Introduction to Its Methodology. 0th ed. Sage
Publications, 2003.
Tom McKlin et al. “Cognitive presence in web-based learning: A content analysis of students’ online
discussions”. In: IT Forum. Vol. 60. 2002.

More Related Content

What's hot

Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word DependenciesAdmixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
David Inouye
 

What's hot (7)

Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) for ...
Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) for ...Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) for ...
Massive Open Online Courses (MOOCs) and Open Educational Resources (OER) for ...
 
Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word DependenciesAdmixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
 
Lilac 2012 Essential information skills for researchers: A collaborative proj...
Lilac 2012 Essential information skills for researchers: A collaborative proj...Lilac 2012 Essential information skills for researchers: A collaborative proj...
Lilac 2012 Essential information skills for researchers: A collaborative proj...
 
Microblogging in the classroom. From information to participation
Microblogging in the classroom.From information to participationMicroblogging in the classroom.From information to participation
Microblogging in the classroom. From information to participation
 
Expert Perceptions of the Feasibility of MOOCs
Expert Perceptions of the Feasibility of MOOCsExpert Perceptions of the Feasibility of MOOCs
Expert Perceptions of the Feasibility of MOOCs
 
EdTech: communicating and learning virtually - Example of a flipped lecture
EdTech: communicating and learning virtually - Example of a flipped lectureEdTech: communicating and learning virtually - Example of a flipped lecture
EdTech: communicating and learning virtually - Example of a flipped lecture
 
LDA Beginner's Tutorial
LDA Beginner's TutorialLDA Beginner's Tutorial
LDA Beginner's Tutorial
 

Viewers also liked

Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Kovanović et al.   2017 - developing a mooc experimentation platform: insight...Kovanović et al.   2017 - developing a mooc experimentation platform: insight...
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Vitomir Kovanovic
 

Viewers also liked (7)

Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Kovanović et al.   2017 - developing a mooc experimentation platform: insight...Kovanović et al.   2017 - developing a mooc experimentation platform: insight...
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
 
MOOCs in the news- A European perspective
MOOCs in the news- A European perspectiveMOOCs in the news- A European perspective
MOOCs in the news- A European perspective
 
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 TutorialTopic Modeling for Learning Analytics Researchers LAK15 Tutorial
Topic Modeling for Learning Analytics Researchers LAK15 Tutorial
 
Penetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimationPenetrating the black box of time-on-task estimation
Penetrating the black box of time-on-task estimation
 
Inquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & OpportunitiesInquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & Opportunities
 
Automated System for Cognitive Presence Coding
Automated System for Cognitive Presence CodingAutomated System for Cognitive Presence Coding
Automated System for Cognitive Presence Coding
 
What does effective online/blended teaching look like?
What does effective online/blended teaching look like?What does effective online/blended teaching look like?
What does effective online/blended teaching look like?
 

Similar to Learning Analytics for Communities of Inquiry

Web Science Session 4 E-Learning.pdf
Web Science Session 4 E-Learning.pdfWeb Science Session 4 E-Learning.pdf
Web Science Session 4 E-Learning.pdf
Stefanie Panke
 
Team Chat: A Technology for Learning
Team Chat: A Technology for LearningTeam Chat: A Technology for Learning
Team Chat: A Technology for Learning
BCcampus
 

Similar to Learning Analytics for Communities of Inquiry (20)

The Chinese National Top Level Courses Project - using Open Educational Resou...
The Chinese National Top Level Courses Project - using Open Educational Resou...The Chinese National Top Level Courses Project - using Open Educational Resou...
The Chinese National Top Level Courses Project - using Open Educational Resou...
 
A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...
 
Humanizing Online Instruction: A MOOC project report
Humanizing Online Instruction: A MOOC project reportHumanizing Online Instruction: A MOOC project report
Humanizing Online Instruction: A MOOC project report
 
The Power of Open: SUNY Open Education Initiatives
The Power of Open: SUNY Open Education InitiativesThe Power of Open: SUNY Open Education Initiatives
The Power of Open: SUNY Open Education Initiatives
 
Learning Analytics: A Messy Research Construct
Learning Analytics: A Messy Research ConstructLearning Analytics: A Messy Research Construct
Learning Analytics: A Messy Research Construct
 
Define massive open online course: results from systematic review of 84 publi...
Define massive open online course: results from systematic review of 84 publi...Define massive open online course: results from systematic review of 84 publi...
Define massive open online course: results from systematic review of 84 publi...
 
IRJET - Investigating the Common Factor of Drop Out based on Learner’s Perspe...
IRJET - Investigating the Common Factor of Drop Out based on Learner’s Perspe...IRJET - Investigating the Common Factor of Drop Out based on Learner’s Perspe...
IRJET - Investigating the Common Factor of Drop Out based on Learner’s Perspe...
 
Web Science Session 4 E-Learning.pdf
Web Science Session 4 E-Learning.pdfWeb Science Session 4 E-Learning.pdf
Web Science Session 4 E-Learning.pdf
 
A Netnography study of MOOC community
A Netnography study of MOOC communityA Netnography study of MOOC community
A Netnography study of MOOC community
 
Influencing the MOOC agenda - analysis of #MOOC Twitter Data
Influencing the MOOC agenda - analysis of #MOOC Twitter Data  Influencing the MOOC agenda - analysis of #MOOC Twitter Data
Influencing the MOOC agenda - analysis of #MOOC Twitter Data
 
The Production of Open Courses as a Transformative Practice: A Case Study of ...
The Production of Open Courses as a Transformative Practice: A Case Study of ...The Production of Open Courses as a Transformative Practice: A Case Study of ...
The Production of Open Courses as a Transformative Practice: A Case Study of ...
 
The production of open courses as a transformative practice: A case study of ...
The production of open courses as a transformative practice: A case study of ...The production of open courses as a transformative practice: A case study of ...
The production of open courses as a transformative practice: A case study of ...
 
Developing information literacy through Web 2.0: a research proposal about t...
Developing information literacy through Web 2.0:  a research proposal about t...Developing information literacy through Web 2.0:  a research proposal about t...
Developing information literacy through Web 2.0: a research proposal about t...
 
Why Institutions Adopt MOOCs: Breaking Down Traditional Barriers or Reproduci...
Why Institutions Adopt MOOCs: Breaking Down Traditional Barriers or Reproduci...Why Institutions Adopt MOOCs: Breaking Down Traditional Barriers or Reproduci...
Why Institutions Adopt MOOCs: Breaking Down Traditional Barriers or Reproduci...
 
Assessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methodsAssessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methods
 
«Implicaciones de datos de aprendizaje procedentes de MOOCs», por Edmundo Tov...
«Implicaciones de datos de aprendizaje procedentes de MOOCs», por Edmundo Tov...«Implicaciones de datos de aprendizaje procedentes de MOOCs», por Edmundo Tov...
«Implicaciones de datos de aprendizaje procedentes de MOOCs», por Edmundo Tov...
 
Methodological innovation for mathematics education research
Methodological innovation for mathematics education researchMethodological innovation for mathematics education research
Methodological innovation for mathematics education research
 
Team Chat: A Technology for Learning
Team Chat: A Technology for LearningTeam Chat: A Technology for Learning
Team Chat: A Technology for Learning
 
SEAL - Second Environment, Advanced Learning
SEAL - Second Environment, Advanced LearningSEAL - Second Environment, Advanced Learning
SEAL - Second Environment, Advanced Learning
 
Learner Centered Environments
Learner Centered EnvironmentsLearner Centered Environments
Learner Centered Environments
 

More from Vitomir Kovanovic

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
Vitomir Kovanovic
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
Vitomir Kovanovic
 

More from Vitomir Kovanovic (13)

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
 
Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...
 
Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...
 
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
 
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
 
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...
 
Unsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics ResearchersUnsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics Researchers
 
Introduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics ResearchersIntroduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics Researchers
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
 
Introduction to Epistemic Network Analysis
Introduction to Epistemic Network AnalysisIntroduction to Epistemic Network Analysis
Introduction to Epistemic Network Analysis
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
 
SFU SIAT Comprehensive Examination
SFU SIAT Comprehensive ExaminationSFU SIAT Comprehensive Examination
SFU SIAT Comprehensive Examination
 

Recently uploaded

Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh +966572737505 get cytotec
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
23050636
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
pwgnohujw
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
a8om7o51
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
ppy8zfkfm
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
yulianti213969
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
jk0tkvfv
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
ju0dztxtn
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Stephen266013
 

Recently uploaded (20)

SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarjSCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
SCI8-Q4-MOD11.pdfwrwujrrjfaajerjrajrrarj
 
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster AnalysisData Analysis Project Presentation : NYC Shooting Cluster Analysis
Data Analysis Project Presentation : NYC Shooting Cluster Analysis
 
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotecAbortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
Abortion pills in Riyadh Saudi Arabia (+966572737505 buy cytotec
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
 
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...Genuine love spell caster )! ,+27834335081)   Ex lover back permanently in At...
Genuine love spell caster )! ,+27834335081) Ex lover back permanently in At...
 
Seven tools of quality control.slideshare
Seven tools of quality control.slideshareSeven tools of quality control.slideshare
Seven tools of quality control.slideshare
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
如何办理加州大学伯克利分校毕业证(UCB毕业证)成绩单留信学历认证
 
Predictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting TechniquesPredictive Precipitation: Advanced Rain Forecasting Techniques
Predictive Precipitation: Advanced Rain Forecasting Techniques
 
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
1:1原版定制利物浦大学毕业证(Liverpool毕业证)成绩单学位证书留信学历认证
 
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
 
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor NetworksSensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
Sensing the Future: Anomaly Detection and Event Prediction in Sensor Networks
 
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
Data Visualization Exploring and Explaining with Data 1st Edition by Camm sol...
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
obat aborsi Bontang wa 081336238223 jual obat aborsi cytotec asli di Bontang6...
 
社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
如何办理英国卡迪夫大学毕业证(Cardiff毕业证书)成绩单留信学历认证
 
Audience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptxAudience Researchndfhcvnfgvgbhujhgfv.pptx
Audience Researchndfhcvnfgvgbhujhgfv.pptx
 

Learning Analytics for Communities of Inquiry

  • 1. Learning Analytics for Communities of Inquiry Vitomir Kovanovi´c1,3 Dragan Gaˇsevi´c1,2 Marek Hatala3 v.kovanovic@ed.ac.uk dgasevic@acm.org mhatala@sfu.ca 1 School of Informatics 2 Moray House School of Education University of Edinburgh University of Edinburgh Edinburgh, United Kingdom Edinburgh, United Kingdom 3 School of Interactive Arts and Technology Simon Fraser University Burnaby, BC, Canada March 17, 2015 Marist College, Poughkeepsie, NY, USA
  • 2. Introduction Ph.D research Overview High-level The goal of my PhD research is to expand the knowldge about inquiry-based digital learning through the development of novel learning analytics models. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 1 / 22
  • 3. Introduction Ph.D research Overview High-level The goal of my PhD research is to expand the knowldge about inquiry-based digital learning through the development of novel learning analytics models. Low-level I’m looking at how use of text analytics, social network analysis and trace data clustering can be used to improve the understanding of the Community of Inquiry (CoI) model of distance education. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 1 / 22
  • 4. Background Inquiry-based online learning Asynchronous online discussions • Primary means of social interaction in online communities, • Frequently used for various types of education delivery (e.g., blended, F2F, distance), • Well supported by the social-constructivist pedagogies, • Can serve multiple purposes (e.g., social support, community building, QA, student-instructor interaction, social learning), • Widely used, often not appropriately, • Require substantial effort from instructors. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 2 / 22
  • 5. Background Community of Inquiry (CoI) Framework Community of Inquiry (CoI) framework Community of Inquiry is a conceptual framework outlying the important constructs that define worthwhile educational experience in distance education setting [3]. • Social presence: relationships and social climate in a community. • Cognitive presence: phases of cognitive engagement and knowledge construction. • Teaching presence: instructional role during social learning. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 3 / 22
  • 6. Background Community of Inquiry (CoI) Framework Community of Inquiry (CoI) framework Community of Inquiry is a conceptual framework outlying the important constructs that define worthwhile educational experience in distance education setting [3]. CoI framework is: • Extensively researched and validated, • Widely used in distance education research. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 3 / 22
  • 7. Background Community of Inquiry (CoI) Framework Cognitive Presence Cognitive Presence “an extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.” [3, p .89] Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 4 / 22
  • 8. Background Community of Inquiry (CoI) Framework Cognitive Presence Cognitive Presence “an extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.” [3, p .89] Four phases of cognitive presence: 1 Triggering event: Some issue, dilemma or problem is identified. 2 Exploration: Students move between private world of reflection and shared world of social knowledge construction. 3 Integration: Students filter irrelevant information and synthesize new knowledge. 4 Resolution: Students analyze practical applicability, test different hypotheses, and start a new learning cycle. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 4 / 22
  • 9. Background Community of Inquiry (CoI) Framework CoI coding instrument • Based on quantitative content analysis (QCA) [11], • Defines three coding schemes for each of the presences, • Use of whole message as unit of analysis, • Look for particular indicators of different sociocognitive processes, • Additional heuristics, • Code-up: When a message clearly displays indicators of several phases it is coded to the latest phase, and • Code-down: When it is not clear which phase is reflected, code to the earliest phase. • Requires expertise with coding instrument and domain knowledge. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 5 / 22
  • 10. Background Community of Inquiry (CoI) Framework CoI coding instrument • Based on quantitative content analysis (QCA) [11], • Defines three coding schemes for each of the presences, • Use of whole message as unit of analysis, • Look for particular indicators of different sociocognitive processes, • Additional heuristics, • Code-up: When a message clearly displays indicators of several phases it is coded to the latest phase, and • Code-down: When it is not clear which phase is reflected, code to the earliest phase. • Requires expertise with coding instrument and domain knowledge. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 5 / 22
  • 11. Background Community of Inquiry (CoI) Framework CoI challenges • Practical challenges, related to its adoption by instructors, • Theoretical challenges, related to current understanding of social interactions, human agency and technology use. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 6 / 22
  • 12. Background Community of Inquiry (CoI) Framework CoI practical challenges • Content analysis instrument: • Time consuming, labor intensive manual message coding, • Hard to scale, typically used for small sample studies, • Typically used for research purposes after courses are over. As a result, content analysis (including CoI) had almost no impact on educational practice [2]. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
  • 13. Background Community of Inquiry (CoI) Framework CoI practical challenges • Content analysis instrument: • Time consuming, labor intensive manual message coding, • Hard to scale, typically used for small sample studies, • Typically used for research purposes after courses are over. As a result, content analysis (including CoI) had almost no impact on educational practice [2]. • Survey instrument: • Removes the need for manual coding, • Administered at the end of the course, • Does not allow any feedback for the duration of the course. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
  • 14. Background Community of Inquiry (CoI) Framework CoI practical challenges • Content analysis instrument: • Time consuming, labor intensive manual message coding, • Hard to scale, typically used for small sample studies, • Typically used for research purposes after courses are over. As a result, content analysis (including CoI) had almost no impact on educational practice [2]. • Survey instrument: • Removes the need for manual coding, • Administered at the end of the course, • Does not allow any feedback for the duration of the course. • A need for more proactive use of available data, • Explanations for observed levels of thee presences, • Suggestions and guidelines for instructors to direct their instructional interventions. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 7 / 22
  • 15. Background Community of Inquiry (CoI) Framework CoI theoretical challenges Although CoI framework is rooted in social constructivism, there is barely any mention of social networks and social capital in current CoI literature: • How does student position in social network reflect on the development of social and cognitive presence and ultimately on learning outcomes? • What types of interactions promote the development of students’ social capital within communities of inquiry? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
  • 16. Background Community of Inquiry (CoI) Framework CoI theoretical challenges Although CoI framework is rooted in social constructivism, there is barely any mention of social networks and social capital in current CoI literature: • How does student position in social network reflect on the development of social and cognitive presence and ultimately on learning outcomes? • What types of interactions promote the development of students’ social capital within communities of inquiry? Human agency and self-regulation of learning are also not frequently discussed: • How does students use available technology to learn in inquiry-based courses? • How does different technology use affects development of cognitive presence, attainment of learning objectives and course performance? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
  • 17. Background Community of Inquiry (CoI) Framework CoI theoretical challenges Although CoI framework is rooted in social constructivism, there is barely any mention of social networks and social capital in current CoI literature: • How does student position in social network reflect on the development of social and cognitive presence and ultimately on learning outcomes? • What types of interactions promote the development of students’ social capital within communities of inquiry? Human agency and self-regulation of learning are also not frequently discussed: • How does students use available technology to learn in inquiry-based courses? • How does different technology use affects development of cognitive presence, attainment of learning objectives and course performance? CoI adoption outside typical small-class DE courses is not widely described: • What are the challenges of CoI use in larger courses (e.g., MOOCs)? • Under what conditions can CoI be used in large online courses? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 8 / 22
  • 18. Research questions Research questions Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 9 / 22
  • 19. Research questions Research questions • How to enable for easier and more scalable quantitative content analysis in accordance with community of inquiry coding schemes? • Which social processes, and to what extent, are indicative of the development of the social capital in communities of inquiry? • What is the relationship between students’ social capital and the development of cognitive presence? • With respect to the development of cognitive presence, are central positions within student social networks beneficial or not? • What are the main technology use profiles withing communities of inquiry? • How does different technology-use profile affect the development of cognitive presence and attainment of learning objectives? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 9 / 22
  • 20. Proposed approach Proposed approach Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
  • 21. Proposed approach Proposed approach Proposed approach Development of learning analytics for communities of inquiry. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
  • 22. Proposed approach Proposed approach Proposed approach Development of learning analytics for communities of inquiry. In general, Build supervised and unsupervised analytical models to address the existing challenges with Community of Inquiry framework. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
  • 23. Proposed approach Proposed approach Proposed approach Development of learning analytics for communities of inquiry. More precisely, 1 Develop a text analytics system for automated message coding in accordance with cognitive presence coding scheme, 2 Develop a predictive model for understanding the relationship between social network capital and social presence, 3 Identify technology-use profiles based on trace data clustering and examine their relationship with development of cognitive presence and learning outcomes. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 10 / 22
  • 24. Proposed approach Text analytics system • Development of a novel text classifier for CoI message coding, • There have been few attempts to automate CoI coding process [12, 1], • Their accuracy not sufficient for practical adoption, • Based on popular classification techniques and surface text features. • Warrants development of novel text classification method that is more suitable for a given problem, • Provides a more detailed operationalization of cognitive presence coding scheme. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 11 / 22
  • 25. Proposed approach Text analytics system: typical approach • Pose a problem as a traditional multi-class supervised classification problem, • Use of surface-level features (N-grams), • Use of part-of-speech (POS) tags, • Use of combinations of N-grams and POS tags (back-off N-grams), • Use of grammatical dependency triplets, • Thread position features, • Standard classification algorithms (SVM, Naive Bayes), • Cross validation a preferred method of validation accuracy and parameters optimization, • In order to be done correctly, requires nesting of cross-validation. • Does not capture code-up and code-down rules, • Issues with quoting of messages, and • Issues with rare classes (i.e., resolution phase). Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 12 / 22
  • 26. Proposed approach Text analytics system: typical approach • Pose a problem as a traditional multi-class supervised classification problem, • Use of surface-level features (N-grams), • Use of part-of-speech (POS) tags, • Use of combinations of N-grams and POS tags (back-off N-grams), • Use of grammatical dependency triplets, • Thread position features, • Standard classification algorithms (SVM, Naive Bayes), • Cross validation a preferred method of validation accuracy and parameters optimization, • In order to be done correctly, requires nesting of cross-validation. • Does not capture code-up and code-down rules, • Issues with quoting of messages, and • Issues with rare classes (i.e., resolution phase). Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 13 / 22
  • 27. Proposed approach Text analytics system: better approach • Take into the account the class (i.e., phase) of the previous message, • Take into the account the phase of cognitive development of a given student, and • Take into the account code-up and code-down rules, • Code-up needs a feature overriding, • Code-down needs an estimate of classification certainty, Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 14 / 22
  • 28. Proposed approach Challenges • What features of discourse are good indicators that can be used for building a classification algorithm? • How to support and leverage the cyclic nature of cognitive presence? • How to model feature precedence? • What features are indicative of the different phases of cognitive presence? • How to build automated text classification system that has accuracy high enough to warrant its practical use? • How to automate as much of message coding as possible while preserving sufficiently high classification accuracy? • How to develop a system that is not a domain dependent? • How to speed up train of the system in new domain? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 15 / 22
  • 29. Proposed approach Social analytics system • Development of learning analytics based on social network analysis of student interactions, • Understand the role of social presence on the development of social capital, • Understand the link between social network position and the development of cognitive presence. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 16 / 22
  • 30. Proposed approach Technology-use profiles • Development of learning analytics based on student trace-data clustering, • Understand the different ways of educational technology-use, • Understand the link between different technology-use profiles and development of cognitive presence, • Understand the link between different technology-use profiles and success in the course. Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 17 / 22
  • 31. Proposed approach Challenges • How to develop a system that is not a domain dependent? • How to speed up train of the system in new domain? • How to identify technology-use profiles? • What forms of social presence are indicative of student social capital? • What trace data to use for profile identification? Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 18 / 22
  • 32. Methodology Methodology 1 Literature review • Educational research • Content analysis in distance education • Community of Inquiry • Social network analysis • Data mining • Text mining • Online discussion and newsgroup classification • Clustering • Graph-based data mining • Quantitative research methods 2 Data collection • Selected Topics in Software Engineering course • UoE E-learning and digital cultures MOOC 3 Development of classification algorithm • Feature extraction • Algorithm development • Validation 4 Technology-use profiling • Trace data conversion and feature extraction • Cluster discovery and validation • Examination of relationships with cognitive presence and learning outcomes 5 Social network analysis • Social graph extraction • Longitudinal analysis of relationships between network centrality and learning outcomes Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 19 / 22
  • 33. Current Progress Current progress 1 Literature review • Educational research • Content analysis in distance education • Community of Inquiry • Social network analysis • Data mining • Text mining • Online discussion and newsgroup classification • Clustering • Graph-based data mining • Quantitative research methods 2 Data collection • Selected Topics in Software Engineering course • UoE E-learning and digital cultures MOOC 3 Development of classification algorithm • Feature extraction • Algorithm development • Validation 4 Technology-use profiling • Trace data conversion and feature extraction • Cluster discovery and validation • Examination of relationships with cognitive presence and learning outcomes 5 Social network analysis • Social graph extraction • Longitudinal analysis of relationships between network centrality and learning outcomes Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 20 / 22
  • 34. Current Progress Publications • Preliminary classifier implementation [6] • End of course CoI SNA analysis [7] • MOOC Research analysis [4] • MOOC News analysis [10] • Time-on-task estimation challenges [9] • Analysis of cognitive presence linguistic properties [5] • JLA doctoral proposal paper [8] Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 21 / 22
  • 35. Expected contributions Benefits of the proposed research: • Based on the empirical evidence, expand current state of understanding of distance education by investigating learners’ interactions with information (i.e., content), technology, instructors and other learners. • Make a foundation on which improvements on the current educational research and practice could be developed. • Provide mode detailed operationalization of the CoI coding instrument by the means of text analytics, • Provide basis for easier adoption of CoI model by the researchers and practitioners, Vitomir Kovanovic (UoE) Learning Analytics for Communities of Inquiry March 17, 2015, Marist College 22 / 22
  • 37. References I Stephen Corich, Kinshuk Hunt, and Lynn Hunt. “Computerised Content Analysis for Measuring Critical Thinking within Discussion Forums”. In: Journal of e-Learning and Knowledge Society 2.1 (2012). Roisin Donnelly and John Gardner. “Content analysis of computer conferencing transcripts”. In: Interactive Learning Environments 19.4 (2011), pp. 303–315. D. Randy Garrison, Terry Anderson, and Walter Archer. “Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education”. In: The Internet and Higher Education 2.2–3 (1999), pp. 87–105. Dragan Gasevic et al. “Where is Research on Massive Open Online Courses Headed? - A data analysis of the MOOC Research Initiative”. In: The International Review of Research in Open and Distance Learning submitted (2014). Sre´cko Joksimovi´c et al. “Psychological characteristics in cognitive presence of communities of inquiry: A linguistic analysis of online discussions”. In: The Internet and Higher Education 22 (2014), pp. 1–10. Vitomir Kovanovic et al. “Automated Content Analysis of Online Discussion Transcripts”. In: Proceedings of the 2014 Learning Analytics and Knowledge (LAK) Conference Workshop on Machine Learning and Learning Analytics. 2014.
  • 38. References II Vitomir Kovanovic et al. “What is the source of social capital? The association between social network position and social presence in communities of inquiry”. In: The first International Workshop on Graph-based Educational Datamining (G-EDM) 2014. 2014. Vitomir Kovanovi´c and Dragan Gaˇsevi´c. “Learning Analytics for Communities of Inquiry”. In: Journal of Learning Analytics in-press (2014). Vitomir Kovanovi´c et al. “Penetrating the Black Box of Time-on-task Estimation”. In: Proceedings of the Fifth International Conference on Learning Analytics and Knowledge (LAK 2015). Best paper award nominee. 2015. Vitomir Kovanovi´c et al. “What public media reveals about MOOCs?” In: British Journal of Educational Technology accepted (2014). Klaus H. Krippendorff. Content Analysis: An Introduction to Its Methodology. 0th ed. Sage Publications, 2003. Tom McKlin et al. “Cognitive presence in web-based learning: A content analysis of students’ online discussions”. In: IT Forum. Vol. 60. 2002.