SlideShare a Scribd company logo
Automated Content Analysis of Discussion Transcripts
Vitomir Kovanovi´c Dragan Gaˇsevi´c
v.kovanovic@ed.ac.uk dgasevic@acm.org
School of Informatics,
University of Edinburgh
Edinburgh, United Kingdom
v.kovanovic@ed.ac.uk
31 Aug 2015,
University of Edinburgh,
United Kingdom
Asynchronous online discussions -
“gold mine of information” (Henri, 1992)
• They are frequently used for all types of
education delivery,
• Their use produced large amount of data
about learning processes,
• Their use is well supported by the
social-constructivist pedagogies.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 1 / 18
Asynchronous online discussions - issues and challenges
• Produced data is used mainly for research after the courses are over,
• Content analysis techniques are complex and time consuming,
• Content analysis had almost no impact on educational practice (Donnelly and
Gardner, 2011),
• There is a need for more proactive use of the data through automation:
• Few attempts for automated content analysis,
• Focus mostly on surface level characteristics, and
• Not based on well established theories of education.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 2 / 18
Overall idea
Overall idea
To examine how we can use text mining for automation of content
analysis of discussion transcripts.
More specifically,
We looked at the automation of content analysis of cognitive
presence, one of the three main components of Community of
Inquiry framework.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 3 / 18
Community of Inquiry (CoI) model
Community of Inquiry model (Garrison, Anderson, and Archer, 1999)
Conceptual framework outlying important constructs that define worthwhile
educational experience in distance education setting.
Three presences:
• Social presence: relationships and social
climate in a community.
• Cognitive presence: phases of cognitive
engagement and knowledge construction.
• Teaching presence: instructional role
during social learning.
CoI model is:
• Extensively researched and validated.
• Adopts Content Analysis for assessment of
presences.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 4 / 18
Community of Inquiry (CoI) model
Community of Inquiry model (Garrison, Anderson, and Archer, 1999)
Conceptual framework outlying important constructs that define worthwhile
educational experience in distance education setting.
Three presences:
• Social presence: relationships and social
climate in a community.
• Cognitive presence: phases of cognitive
engagement and knowledge construction.
• Teaching presence: instructional role
during social learning.
CoI model is:
• Extensively researched and validated.
• Adopts Content Analysis for assessment of
presences.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 4 / 18
Cognitive presence
Cognitive Presence
“an extent to which the participants in any particular configuration of a
community of inquiry are able to construct meaning through sustained
communication.” (Garrison, Anderson, and Archer, 1999, p .89)
Four phases of cognitive presence:
1 Triggering event: Some issue, dilemma or problem is identified.
2 Exploration: Students move between private world of reflection and shared
world of social knowledge construction.
3 Integration: Students filter irrelevant information and synthesize new
knowledge.
4 Resolution: Students analyze practical applicability, test different
hypotheses, and start a new learning cycle.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 5 / 18
Cognitive presence coding scheme
• Use of whole message as unit of analysis,
• Look for particular indicators of different sociocognitive processes,
• Requires expertise with coding instrument and domain knowledge.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 6 / 18
Community of Inquiry (CoI) model
Issues and challenges:
• Very labor intensive,
• Crude coding scheme,
• Requires experienced coders,
• Can’t be used for real-time monitoring,
• Not explaining reasons behind observed levels of presences, and
• Not providing suggestions and guidelines for instructors to direct their
pedagogical decisions.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 7 / 18
Data set
• Six offerings of graduate level course in software engineering.
• Total of 1747 messages, 81 students,
• Manually coded by two coders (agreement = 98.1%, Cohen’s κ = 0.974),
ID Phase Messages (%)
0 Other 140 8.01%
1 Triggering Event 308 17.63%
2 Exploration 684 39.17%
3 Integration 508 29.08%
4 Resolution 107 6.12%
All phases 1747 100%
Number of Messages in Different Phases of Cognitive Presence
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 8 / 18
Feature extraction
• Unigrams, Bigrams and Trigrams,
• Part-of-Speech Bigrams and Trigrams,
• Backoff Bigrams and Trigrams:
Example: “John is working.”
Bigrams:
• john is,
• is working.
Backoff Bigrams:
• john verb ,
• noun is,
• is verb
• verb working.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 9 / 18
Feature extraction
• Dependency triplets: rel, head, modifier
Example: “Bills on ports and immigration were submitted by Senator
Brownback, Republican of Kansas.”
nsubjpass, submitted, Bills
auxpass, submitted, were
agent, submitted, Brownback
nn, Brownback, Senator
appos, Brownback, Republican
prep of, Republican, Kansas
prep on, Bills, ports
conj and, ports, immigration
prep on, Bills, immigration
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 10 / 18
Feature extraction
• Backoff dependency triplets:
Example: “Bills on ports and immigration were submitted by Senator
Brownback, Republican of Kansas.”
Dependency triplet:
• conj and, ports, immigration
Backoff dependency triplets:
• conj and, noun , immigration
• conj and, ports, noun
• conj and, noun , noun
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 11 / 18
Additional features
• Number of named entities in the message
Brainstorming should involve more concepts than posing a question,
• Is message first in the discussion?
Posing questions is more likely to be initiating discussions,
• Is message a reply to the first message in the discussion?
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 12 / 18
Classification
Classifier:
• SVM classifier with RBF kernel.
• Accuracy and kernel parameter tuning evaluated using nested 5-fold
cross-validation.
• Only features with support of 10 or more,
• Accuracy evaluated using 10 fold cross-validation,
• Comparison of models using McNemar’s test.
Implementation:
• Implemented in Java,
• Feature extraction using Stanford CoreNLP1
toolkit,
• Tokenization, Part-of-Speech, and Dependency parsing modules
• Classification using Weka (Witten, Frank, and Hall, 2011) and
LibSVM (Chang and Lin, 2011), and
• Statistical comparison using Java Statistical Classes (JSC)2
1http://nlp.stanford.edu/software/corenlp.shtml
2http://www.jsc.nildram.co.uk/index.htm
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 13 / 18
Results
• We achieved Cohen’s κ of 0.42 for our classification problem.
• Better then the existing Neural Network system (Cohen’s κ=0.31).
• Unigram baseline model achieved Cohen’s κ of 0.33.
Error analysis:
Predicted
Actual Other Trigg. Expl. Integ. Resol.
Other 17 04 05 02 00
Triggering 01 42 ⇒1
14 03 01
Exploration 02 09 98 24 04
Integration 01 03 38 ⇐1,2
56 04
Resolution 00 00 03 15 ⇐2
03
Confusion Matrix
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 14 / 18
Challenges
1 Effect of the large relative size of the
exploration class,
2 Effect of the code-up rule for coding,
3 No relative importance of features, and
4 Context is not taken into the account.
Code-up rule for coding
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 15 / 18
In progress: making use of tread context
• Discussions (and students’ learning) progresses from triggering to resolutions.
• Content of a message depends on the content of the previous messages.
• Content of a message depends on the learning progress of a given student.
Model for message classification
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 16 / 18
Approach: Hidden Markov models (HMMs) & Conditional
random fields (CRFs)
• Hidden Markov Models:
• HMMs used to models system states and their transitions in a variety of
contexts.
• Widely used, Bayesian Knowledge Tracing models based on HMMs.
• Challenges with HMM:
• Can this be modeled as HMM (2nd order HMMs?)
• Dependency only on a single previous state,
• One manifest variable for each state
• Conditional random fields:
• Used for structured predictions (e.g., speech recognition)
• For speech recognition, take into the account the classes of all letters in a word.
• Widely used in natural language processing,
• More flexible than HMMs,
• Challenges with CRF:
• Too many parameters to estimate with little data
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 17 / 18
Conclusions and future work
Summary:
• Promising path to explore,
• Use of backoff trigrams, plain and backoff dependency triplets, entity count
and first message indicator seems useful,
Future work:
• Additional types of features which look at the context of previous messages
(e.g., convergence vs. divergence),
• Moving away from SVM, explore other classification methods which are
better at explanation
• Give associated probabilities for each classification,
• Give relative importance of different features.
Challenges:
• Challenges with message unit of analysis and surface-level features,
• Low frequency of resolution messages.
V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 18 / 18
Thank you
Vitomir Kovanovic
v.kovanovic@ed.ac.uk
References I
Chang, Chih-Chung and Chih-Jen Lin (2011). “LIBSVM: A library for support vector machines”. In:
ACM Transactions on Intelligent Systems and Technology 2 (3), 27:1–27:27.
Donnelly, Roisin and John Gardner (2011). “Content analysis of computer conferencing transcripts”.
In: Interactive Learning Environments 19.4, pp. 303–315.
Garrison, D. Randy, Terry Anderson, and Walter Archer (1999). “Critical Inquiry in a Text-Based
Environment: Computer Conferencing in Higher Education”. In: The Internet and Higher Education
2.2–3, pp. 87–105.
Henri, France (1992). “Computer Conferencing and Content Analysis”. en. In: Collaborative Learning
Through Computer Conferencing, pp. 117–136.
Witten, Ian H., Eibe Frank, and Mark A. Hall (2011). Data Mining: Practical Machine Learning Tools
and Techniques, Third Edition. 3rd ed. Morgan Kaufmann.

More Related Content

What's hot

Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word DependenciesAdmixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
David Inouye
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Sebastian Ruder
 
LDA Beginner's Tutorial
LDA Beginner's TutorialLDA Beginner's Tutorial
LDA Beginner's Tutorial
Wayne Lee
 
A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...
Vitomir Kovanovic
 
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
cahafner
 
Introduction to OpenSemcq
Introduction to OpenSemcqIntroduction to OpenSemcq
Introduction to OpenSemcq
mbtosic
 
How useful are semantic links for the detection of implicit references in csc...
How useful are semantic links for the detection of implicit references in csc...How useful are semantic links for the detection of implicit references in csc...
How useful are semantic links for the detection of implicit references in csc...
Traian Rebedea
 
Preservice Teachers' Dispositions Toward Technology Integration
Preservice Teachers' Dispositions Toward Technology IntegrationPreservice Teachers' Dispositions Toward Technology Integration
Preservice Teachers' Dispositions Toward Technology Integration
Joan E. Hughes, Ph.D.
 
Research on Recommender Systems: Beyond Ratings and Lists
Research on Recommender Systems: Beyond Ratings and ListsResearch on Recommender Systems: Beyond Ratings and Lists
Research on Recommender Systems: Beyond Ratings and Lists
Denis Parra Santander
 
Commonsense knowledge for Machine Intelligence - part 1
Commonsense knowledge for Machine Intelligence - part 1Commonsense knowledge for Machine Intelligence - part 1
Commonsense knowledge for Machine Intelligence - part 1
Niket Tandon
 
E-Learning Adoption in a Higher Education Setting: An Empirical Study
E-Learning Adoption in a Higher Education Setting: An Empirical StudyE-Learning Adoption in a Higher Education Setting: An Empirical Study
E-Learning Adoption in a Higher Education Setting: An Empirical Study
Dagmar Monett
 
What questions are MOOCs asking? An evidence based investigation
What questions are MOOCs asking? An evidence based investigationWhat questions are MOOCs asking? An evidence based investigation
What questions are MOOCs asking? An evidence based investigation
Eamon Costello
 
Statistical and Empirical Approaches to Spoken Dialog Systems
Statistical and Empirical Approaches to Spoken Dialog SystemsStatistical and Empirical Approaches to Spoken Dialog Systems
Statistical and Empirical Approaches to Spoken Dialog Systems
butest
 
Relevance based ranking of video comments on YouTube
Relevance based ranking of video comments on YouTubeRelevance based ranking of video comments on YouTube
Relevance based ranking of video comments on YouTube
Traian Rebedea
 
Domain Modeling for Personalized Learning
Domain Modeling for Personalized LearningDomain Modeling for Personalized Learning
Domain Modeling for Personalized Learning
Peter Brusilovsky
 
Fuschi current Research and Developments
Fuschi current Research and DevelopmentsFuschi current Research and Developments
Fuschi current Research and Developments
David Luigi FUSCHI Ph.D.
 
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
Yuwei Lin
 
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
Dagmar Monett
 
Assignment 2 powerpoint 2
Assignment 2 powerpoint 2Assignment 2 powerpoint 2
Assignment 2 powerpoint 2
robynwood
 
From Conventional to Technology-Assisted Alternative Assessment for Effective...
From Conventional to Technology-Assisted Alternative Assessment for Effective...From Conventional to Technology-Assisted Alternative Assessment for Effective...
From Conventional to Technology-Assisted Alternative Assessment for Effective...
English Literature and Language Review ELLR
 

What's hot (20)

Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word DependenciesAdmixture of Poisson MRFs: A New Topic Model with Word Dependencies
Admixture of Poisson MRFs: A New Topic Model with Word Dependencies
 
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
Dynamic Topic Modeling via Non-negative Matrix Factorization (Dr. Derek Greene)
 
LDA Beginner's Tutorial
LDA Beginner's TutorialLDA Beginner's Tutorial
LDA Beginner's Tutorial
 
A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...A novel model of cognitive presence assessment using automated learning analy...
A novel model of cognitive presence assessment using automated learning analy...
 
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
Asia-Pacific LSP & Professional Communication, 2017: Developing research meth...
 
Introduction to OpenSemcq
Introduction to OpenSemcqIntroduction to OpenSemcq
Introduction to OpenSemcq
 
How useful are semantic links for the detection of implicit references in csc...
How useful are semantic links for the detection of implicit references in csc...How useful are semantic links for the detection of implicit references in csc...
How useful are semantic links for the detection of implicit references in csc...
 
Preservice Teachers' Dispositions Toward Technology Integration
Preservice Teachers' Dispositions Toward Technology IntegrationPreservice Teachers' Dispositions Toward Technology Integration
Preservice Teachers' Dispositions Toward Technology Integration
 
Research on Recommender Systems: Beyond Ratings and Lists
Research on Recommender Systems: Beyond Ratings and ListsResearch on Recommender Systems: Beyond Ratings and Lists
Research on Recommender Systems: Beyond Ratings and Lists
 
Commonsense knowledge for Machine Intelligence - part 1
Commonsense knowledge for Machine Intelligence - part 1Commonsense knowledge for Machine Intelligence - part 1
Commonsense knowledge for Machine Intelligence - part 1
 
E-Learning Adoption in a Higher Education Setting: An Empirical Study
E-Learning Adoption in a Higher Education Setting: An Empirical StudyE-Learning Adoption in a Higher Education Setting: An Empirical Study
E-Learning Adoption in a Higher Education Setting: An Empirical Study
 
What questions are MOOCs asking? An evidence based investigation
What questions are MOOCs asking? An evidence based investigationWhat questions are MOOCs asking? An evidence based investigation
What questions are MOOCs asking? An evidence based investigation
 
Statistical and Empirical Approaches to Spoken Dialog Systems
Statistical and Empirical Approaches to Spoken Dialog SystemsStatistical and Empirical Approaches to Spoken Dialog Systems
Statistical and Empirical Approaches to Spoken Dialog Systems
 
Relevance based ranking of video comments on YouTube
Relevance based ranking of video comments on YouTubeRelevance based ranking of video comments on YouTube
Relevance based ranking of video comments on YouTube
 
Domain Modeling for Personalized Learning
Domain Modeling for Personalized LearningDomain Modeling for Personalized Learning
Domain Modeling for Personalized Learning
 
Fuschi current Research and Developments
Fuschi current Research and DevelopmentsFuschi current Research and Developments
Fuschi current Research and Developments
 
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
Some Methodological Thoughts on Using Text Mining for Frame Analysis of Media...
 
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
Evolving Lesson Plans to Assist Educators: From Paper-Based to Adaptive Lesso...
 
Assignment 2 powerpoint 2
Assignment 2 powerpoint 2Assignment 2 powerpoint 2
Assignment 2 powerpoint 2
 
From Conventional to Technology-Assisted Alternative Assessment for Effective...
From Conventional to Technology-Assisted Alternative Assessment for Effective...From Conventional to Technology-Assisted Alternative Assessment for Effective...
From Conventional to Technology-Assisted Alternative Assessment for Effective...
 

Viewers also liked

Inquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & OpportunitiesInquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & Opportunities
Vitomir Kovanovic
 
Learning Analytics for Communities of Inquiry
Learning Analytics for Communities of InquiryLearning Analytics for Communities of Inquiry
Learning Analytics for Communities of Inquiry
Vitomir Kovanovic
 
MOOCs in the news- A European perspective
MOOCs in the news- A European perspectiveMOOCs in the news- A European perspective
MOOCs in the news- A European perspective
Vitomir Kovanovic
 
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Kovanović et al.   2017 - developing a mooc experimentation platform: insight...Kovanović et al.   2017 - developing a mooc experimentation platform: insight...
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Vitomir Kovanovic
 
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
Vitomir Kovanovic
 
MOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunitiesMOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunities
Vitomir Kovanovic
 
What does effective online/blended teaching look like?
What does effective online/blended teaching look like?What does effective online/blended teaching look like?
What does effective online/blended teaching look like?
Vitomir Kovanovic
 
Automated System for Cognitive Presence Coding
Automated System for Cognitive Presence CodingAutomated System for Cognitive Presence Coding
Automated System for Cognitive Presence Coding
Vitomir Kovanovic
 

Viewers also liked (8)

Inquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & OpportunitiesInquiry-based Learning & MOOCs: Challenges & Opportunities
Inquiry-based Learning & MOOCs: Challenges & Opportunities
 
Learning Analytics for Communities of Inquiry
Learning Analytics for Communities of InquiryLearning Analytics for Communities of Inquiry
Learning Analytics for Communities of Inquiry
 
MOOCs in the news- A European perspective
MOOCs in the news- A European perspectiveMOOCs in the news- A European perspective
MOOCs in the news- A European perspective
 
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
Kovanović et al.   2017 - developing a mooc experimentation platform: insight...Kovanović et al.   2017 - developing a mooc experimentation platform: insight...
Kovanović et al. 2017 - developing a mooc experimentation platform: insight...
 
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
A Novel Model of Cognitive Presence Assessment Using Automated Learning Analy...
 
MOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunitiesMOOCs & Social Learning: Challenges and opportunities
MOOCs & Social Learning: Challenges and opportunities
 
What does effective online/blended teaching look like?
What does effective online/blended teaching look like?What does effective online/blended teaching look like?
What does effective online/blended teaching look like?
 
Automated System for Cognitive Presence Coding
Automated System for Cognitive Presence CodingAutomated System for Cognitive Presence Coding
Automated System for Cognitive Presence Coding
 

Similar to Automated Content Analysis of Discussion Transcripts

Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
Lifeng (Aaron) Han
 
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar PosterCritiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
Mark Guzdial
 
Sybrandt Thesis Proposal Presentation
Sybrandt Thesis Proposal PresentationSybrandt Thesis Proposal Presentation
Sybrandt Thesis Proposal Presentation
Justin Sybrandt, Ph.D.
 
Statistical Machine Learning - Lecture 1 - ANU
Statistical Machine Learning - Lecture 1 - ANUStatistical Machine Learning - Lecture 1 - ANU
Statistical Machine Learning - Lecture 1 - ANU
mthmlcvanu
 
ISWC2023-McGuinnessTWC16x9FinalShort.pdf
ISWC2023-McGuinnessTWC16x9FinalShort.pdfISWC2023-McGuinnessTWC16x9FinalShort.pdf
ISWC2023-McGuinnessTWC16x9FinalShort.pdf
Deborah McGuinness
 
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
Tom Mens
 
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
Hironori Washizaki
 
Past and Future of Software Testing and Analysis
Past and Future of Software Testing and AnalysisPast and Future of Software Testing and Analysis
Past and Future of Software Testing and Analysis
Lionel Briand
 
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdfShibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
Shibani22
 
Software Engineering Research: Leading a Double-Agent Life.
Software Engineering Research: Leading a Double-Agent Life.Software Engineering Research: Leading a Double-Agent Life.
Software Engineering Research: Leading a Double-Agent Life.
Lionel Briand
 
Summarization Techniques for Code, Change, Testing and User Feedback
Summarization Techniques for Code, Change, Testing and User FeedbackSummarization Techniques for Code, Change, Testing and User Feedback
Summarization Techniques for Code, Change, Testing and User Feedback
Sebastiano Panichella
 
Tutorial on Question Answering Systems
Tutorial on Question Answering Systems Tutorial on Question Answering Systems
Tutorial on Question Answering Systems
Saeedeh Shekarpour
 
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
Sunghwan Kim
 
Determining the Credibility of Science Communication
Determining the Credibility of Science CommunicationDetermining the Credibility of Science Communication
Determining the Credibility of Science Communication
Isabelle Augenstein
 
cv
cvcv
Big Qualitative Data, Big Team, Little Time - A Path to Publication
Big Qualitative Data, Big Team, Little Time - A Path to PublicationBig Qualitative Data, Big Team, Little Time - A Path to Publication
Big Qualitative Data, Big Team, Little Time - A Path to Publication
QSR International
 
Profiling Linked Open Data
Profiling Linked Open DataProfiling Linked Open Data
Profiling Linked Open Data
Blerina Spahiu
 
Social Phrases Having Impact in Altmetrics - SOPHIA
Social Phrases Having Impact in Altmetrics - SOPHIASocial Phrases Having Impact in Altmetrics - SOPHIA
Social Phrases Having Impact in Altmetrics - SOPHIA
Insight_Altmetrics
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Matthew Lease
 
Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)
evabl444
 

Similar to Automated Content Analysis of Discussion Transcripts (20)

Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
Meta-Evaluation of Translation Evaluation Methods: a systematic up-to-date ov...
 
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar PosterCritiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar Poster
 
Sybrandt Thesis Proposal Presentation
Sybrandt Thesis Proposal PresentationSybrandt Thesis Proposal Presentation
Sybrandt Thesis Proposal Presentation
 
Statistical Machine Learning - Lecture 1 - ANU
Statistical Machine Learning - Lecture 1 - ANUStatistical Machine Learning - Lecture 1 - ANU
Statistical Machine Learning - Lecture 1 - ANU
 
ISWC2023-McGuinnessTWC16x9FinalShort.pdf
ISWC2023-McGuinnessTWC16x9FinalShort.pdfISWC2023-McGuinnessTWC16x9FinalShort.pdf
ISWC2023-McGuinnessTWC16x9FinalShort.pdf
 
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
ICSME 2016 keynote: An ecosystemic and socio-technical view on software maint...
 
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
Rubric-based Assessment of Programming Thinking Skills and Comparative Evalua...
 
Past and Future of Software Testing and Analysis
Past and Future of Software Testing and AnalysisPast and Future of Software Testing and Analysis
Past and Future of Software Testing and Analysis
 
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdfShibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
Shibani Antonette_Augmenting pedagogic writing practice with CLAD.pdf
 
Software Engineering Research: Leading a Double-Agent Life.
Software Engineering Research: Leading a Double-Agent Life.Software Engineering Research: Leading a Double-Agent Life.
Software Engineering Research: Leading a Double-Agent Life.
 
Summarization Techniques for Code, Change, Testing and User Feedback
Summarization Techniques for Code, Change, Testing and User FeedbackSummarization Techniques for Code, Change, Testing and User Feedback
Summarization Techniques for Code, Change, Testing and User Feedback
 
Tutorial on Question Answering Systems
Tutorial on Question Answering Systems Tutorial on Question Answering Systems
Tutorial on Question Answering Systems
 
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
Cheminformatics Online Chemistry Course (OLCC): A Community Effort to Introdu...
 
Determining the Credibility of Science Communication
Determining the Credibility of Science CommunicationDetermining the Credibility of Science Communication
Determining the Credibility of Science Communication
 
cv
cvcv
cv
 
Big Qualitative Data, Big Team, Little Time - A Path to Publication
Big Qualitative Data, Big Team, Little Time - A Path to PublicationBig Qualitative Data, Big Team, Little Time - A Path to Publication
Big Qualitative Data, Big Team, Little Time - A Path to Publication
 
Profiling Linked Open Data
Profiling Linked Open DataProfiling Linked Open Data
Profiling Linked Open Data
 
Social Phrases Having Impact in Altmetrics - SOPHIA
Social Phrases Having Impact in Altmetrics - SOPHIASocial Phrases Having Impact in Altmetrics - SOPHIA
Social Phrases Having Impact in Altmetrics - SOPHIA
 
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & OpportunitiesDeep Learning for Information Retrieval: Models, Progress, & Opportunities
Deep Learning for Information Retrieval: Models, Progress, & Opportunities
 
Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)Experimenting with eXtreme Design (EKAW2010)
Experimenting with eXtreme Design (EKAW2010)
 

More from Vitomir Kovanovic

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
Vitomir Kovanovic
 
Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...
Vitomir Kovanovic
 
Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...
Vitomir Kovanovic
 
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Vitomir Kovanovic
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
Vitomir Kovanovic
 
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Vitomir Kovanovic
 
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Vitomir Kovanovic
 
Unsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics ResearchersUnsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics Researchers
Vitomir Kovanovic
 
Introduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics ResearchersIntroduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics Researchers
Vitomir Kovanovic
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
Vitomir Kovanovic
 
Introduction to Epistemic Network Analysis
Introduction to Epistemic Network AnalysisIntroduction to Epistemic Network Analysis
Introduction to Epistemic Network Analysis
Vitomir Kovanovic
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
Vitomir Kovanovic
 
Assessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methodsAssessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methods
Vitomir Kovanovic
 

More from Vitomir Kovanovic (13)

Introduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and ManagersIntroduction to Learning Analytics for High School Teachers and Managers
Introduction to Learning Analytics for High School Teachers and Managers
 
Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...Extending video interactions to support self-regulated learning in an online ...
Extending video interactions to support self-regulated learning in an online ...
 
Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...Analysing social presence in online discussions through network and text anal...
Analysing social presence in online discussions through network and text anal...
 
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
Automated Analysis of Cognitive Presence in Online Discussions Written in Por...
 
Validating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analyticsValidating a theorized model of engagement in learning analytics
Validating a theorized model of engagement in learning analytics
 
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
Examining the Value of Learning Analytics for Supporting Work-integrated Lear...
 
Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...Developing Self-regulated Learning in High-school Students: The Role of Learn...
Developing Self-regulated Learning in High-school Students: The Role of Learn...
 
Unsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics ResearchersUnsupervised Learning for Learning Analytics Researchers
Unsupervised Learning for Learning Analytics Researchers
 
Introduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics ResearchersIntroduction to R for Learning Analytics Researchers
Introduction to R for Learning Analytics Researchers
 
Introduction to Learning Analytics
Introduction to Learning AnalyticsIntroduction to Learning Analytics
Introduction to Learning Analytics
 
Introduction to Epistemic Network Analysis
Introduction to Epistemic Network AnalysisIntroduction to Epistemic Network Analysis
Introduction to Epistemic Network Analysis
 
Understand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analyticsUnderstand students’ self-reflections through learning analytics
Understand students’ self-reflections through learning analytics
 
Assessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methodsAssessing cognitive presence using automated learning analytics methods
Assessing cognitive presence using automated learning analytics methods
 

Recently uploaded

Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
PsychoTech Services
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
iammrhaywood
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
Jyoti Chand
 
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdfREASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
giancarloi8888
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
melliereed
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
zuzanka
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
zuzanka
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
Nicholas Montgomery
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
Nguyen Thanh Tu Collection
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
Celine George
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
HajraNaeem15
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
deepaannamalai16
 
A Visual Guide to 1 Samuel | A Tale of Two Hearts
A Visual Guide to 1 Samuel | A Tale of Two HeartsA Visual Guide to 1 Samuel | A Tale of Two Hearts
A Visual Guide to 1 Samuel | A Tale of Two Hearts
Steve Thomason
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
EduSkills OECD
 
Bonku-Babus-Friend by Sathyajith Ray (9)
Bonku-Babus-Friend by Sathyajith Ray  (9)Bonku-Babus-Friend by Sathyajith Ray  (9)
Bonku-Babus-Friend by Sathyajith Ray (9)
nitinpv4ai
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
GeorgeMilliken2
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDFLifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Vivekanand Anglo Vedic Academy
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
TechSoup
 

Recently uploaded (20)

Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...Gender and Mental Health - Counselling and Family Therapy Applications and In...
Gender and Mental Health - Counselling and Family Therapy Applications and In...
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptxNEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
NEWSPAPERS - QUESTION 1 - REVISION POWERPOINT.pptx
 
Wound healing PPT
Wound healing PPTWound healing PPT
Wound healing PPT
 
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdfREASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
REASIGNACION 2024 UGEL CHUPACA 2024 UGEL CHUPACA.pdf
 
Nutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour TrainingNutrition Inc FY 2024, 4 - Hour Training
Nutrition Inc FY 2024, 4 - Hour Training
 
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptxRESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
RESULTS OF THE EVALUATION QUESTIONNAIRE.pptx
 
SWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptxSWOT analysis in the project Keeping the Memory @live.pptx
SWOT analysis in the project Keeping the Memory @live.pptx
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
 
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
BÀI TẬP DẠY THÊM TIẾNG ANH LỚP 7 CẢ NĂM FRIENDS PLUS SÁCH CHÂN TRỜI SÁNG TẠO ...
 
How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17How to Make a Field Mandatory in Odoo 17
How to Make a Field Mandatory in Odoo 17
 
How to deliver Powerpoint Presentations.pptx
How to deliver Powerpoint  Presentations.pptxHow to deliver Powerpoint  Presentations.pptx
How to deliver Powerpoint Presentations.pptx
 
HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.HYPERTENSION - SLIDE SHARE PRESENTATION.
HYPERTENSION - SLIDE SHARE PRESENTATION.
 
A Visual Guide to 1 Samuel | A Tale of Two Hearts
A Visual Guide to 1 Samuel | A Tale of Two HeartsA Visual Guide to 1 Samuel | A Tale of Two Hearts
A Visual Guide to 1 Samuel | A Tale of Two Hearts
 
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptxBeyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
Beyond Degrees - Empowering the Workforce in the Context of Skills-First.pptx
 
Bonku-Babus-Friend by Sathyajith Ray (9)
Bonku-Babus-Friend by Sathyajith Ray  (9)Bonku-Babus-Friend by Sathyajith Ray  (9)
Bonku-Babus-Friend by Sathyajith Ray (9)
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDFLifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
Lifelines of National Economy chapter for Class 10 STUDY MATERIAL PDF
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
 

Automated Content Analysis of Discussion Transcripts

  • 1. Automated Content Analysis of Discussion Transcripts Vitomir Kovanovi´c Dragan Gaˇsevi´c v.kovanovic@ed.ac.uk dgasevic@acm.org School of Informatics, University of Edinburgh Edinburgh, United Kingdom v.kovanovic@ed.ac.uk 31 Aug 2015, University of Edinburgh, United Kingdom
  • 2. Asynchronous online discussions - “gold mine of information” (Henri, 1992) • They are frequently used for all types of education delivery, • Their use produced large amount of data about learning processes, • Their use is well supported by the social-constructivist pedagogies. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 1 / 18
  • 3. Asynchronous online discussions - issues and challenges • Produced data is used mainly for research after the courses are over, • Content analysis techniques are complex and time consuming, • Content analysis had almost no impact on educational practice (Donnelly and Gardner, 2011), • There is a need for more proactive use of the data through automation: • Few attempts for automated content analysis, • Focus mostly on surface level characteristics, and • Not based on well established theories of education. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 2 / 18
  • 4. Overall idea Overall idea To examine how we can use text mining for automation of content analysis of discussion transcripts. More specifically, We looked at the automation of content analysis of cognitive presence, one of the three main components of Community of Inquiry framework. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 3 / 18
  • 5. Community of Inquiry (CoI) model Community of Inquiry model (Garrison, Anderson, and Archer, 1999) Conceptual framework outlying important constructs that define worthwhile educational experience in distance education setting. Three presences: • Social presence: relationships and social climate in a community. • Cognitive presence: phases of cognitive engagement and knowledge construction. • Teaching presence: instructional role during social learning. CoI model is: • Extensively researched and validated. • Adopts Content Analysis for assessment of presences. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 4 / 18
  • 6. Community of Inquiry (CoI) model Community of Inquiry model (Garrison, Anderson, and Archer, 1999) Conceptual framework outlying important constructs that define worthwhile educational experience in distance education setting. Three presences: • Social presence: relationships and social climate in a community. • Cognitive presence: phases of cognitive engagement and knowledge construction. • Teaching presence: instructional role during social learning. CoI model is: • Extensively researched and validated. • Adopts Content Analysis for assessment of presences. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 4 / 18
  • 7. Cognitive presence Cognitive Presence “an extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.” (Garrison, Anderson, and Archer, 1999, p .89) Four phases of cognitive presence: 1 Triggering event: Some issue, dilemma or problem is identified. 2 Exploration: Students move between private world of reflection and shared world of social knowledge construction. 3 Integration: Students filter irrelevant information and synthesize new knowledge. 4 Resolution: Students analyze practical applicability, test different hypotheses, and start a new learning cycle. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 5 / 18
  • 8. Cognitive presence coding scheme • Use of whole message as unit of analysis, • Look for particular indicators of different sociocognitive processes, • Requires expertise with coding instrument and domain knowledge. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 6 / 18
  • 9. Community of Inquiry (CoI) model Issues and challenges: • Very labor intensive, • Crude coding scheme, • Requires experienced coders, • Can’t be used for real-time monitoring, • Not explaining reasons behind observed levels of presences, and • Not providing suggestions and guidelines for instructors to direct their pedagogical decisions. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 7 / 18
  • 10. Data set • Six offerings of graduate level course in software engineering. • Total of 1747 messages, 81 students, • Manually coded by two coders (agreement = 98.1%, Cohen’s κ = 0.974), ID Phase Messages (%) 0 Other 140 8.01% 1 Triggering Event 308 17.63% 2 Exploration 684 39.17% 3 Integration 508 29.08% 4 Resolution 107 6.12% All phases 1747 100% Number of Messages in Different Phases of Cognitive Presence V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 8 / 18
  • 11. Feature extraction • Unigrams, Bigrams and Trigrams, • Part-of-Speech Bigrams and Trigrams, • Backoff Bigrams and Trigrams: Example: “John is working.” Bigrams: • john is, • is working. Backoff Bigrams: • john verb , • noun is, • is verb • verb working. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 9 / 18
  • 12. Feature extraction • Dependency triplets: rel, head, modifier Example: “Bills on ports and immigration were submitted by Senator Brownback, Republican of Kansas.” nsubjpass, submitted, Bills auxpass, submitted, were agent, submitted, Brownback nn, Brownback, Senator appos, Brownback, Republican prep of, Republican, Kansas prep on, Bills, ports conj and, ports, immigration prep on, Bills, immigration V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 10 / 18
  • 13. Feature extraction • Backoff dependency triplets: Example: “Bills on ports and immigration were submitted by Senator Brownback, Republican of Kansas.” Dependency triplet: • conj and, ports, immigration Backoff dependency triplets: • conj and, noun , immigration • conj and, ports, noun • conj and, noun , noun V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 11 / 18
  • 14. Additional features • Number of named entities in the message Brainstorming should involve more concepts than posing a question, • Is message first in the discussion? Posing questions is more likely to be initiating discussions, • Is message a reply to the first message in the discussion? V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 12 / 18
  • 15. Classification Classifier: • SVM classifier with RBF kernel. • Accuracy and kernel parameter tuning evaluated using nested 5-fold cross-validation. • Only features with support of 10 or more, • Accuracy evaluated using 10 fold cross-validation, • Comparison of models using McNemar’s test. Implementation: • Implemented in Java, • Feature extraction using Stanford CoreNLP1 toolkit, • Tokenization, Part-of-Speech, and Dependency parsing modules • Classification using Weka (Witten, Frank, and Hall, 2011) and LibSVM (Chang and Lin, 2011), and • Statistical comparison using Java Statistical Classes (JSC)2 1http://nlp.stanford.edu/software/corenlp.shtml 2http://www.jsc.nildram.co.uk/index.htm V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 13 / 18
  • 16. Results • We achieved Cohen’s κ of 0.42 for our classification problem. • Better then the existing Neural Network system (Cohen’s κ=0.31). • Unigram baseline model achieved Cohen’s κ of 0.33. Error analysis: Predicted Actual Other Trigg. Expl. Integ. Resol. Other 17 04 05 02 00 Triggering 01 42 ⇒1 14 03 01 Exploration 02 09 98 24 04 Integration 01 03 38 ⇐1,2 56 04 Resolution 00 00 03 15 ⇐2 03 Confusion Matrix V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 14 / 18
  • 17. Challenges 1 Effect of the large relative size of the exploration class, 2 Effect of the code-up rule for coding, 3 No relative importance of features, and 4 Context is not taken into the account. Code-up rule for coding V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 15 / 18
  • 18. In progress: making use of tread context • Discussions (and students’ learning) progresses from triggering to resolutions. • Content of a message depends on the content of the previous messages. • Content of a message depends on the learning progress of a given student. Model for message classification V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 16 / 18
  • 19. Approach: Hidden Markov models (HMMs) & Conditional random fields (CRFs) • Hidden Markov Models: • HMMs used to models system states and their transitions in a variety of contexts. • Widely used, Bayesian Knowledge Tracing models based on HMMs. • Challenges with HMM: • Can this be modeled as HMM (2nd order HMMs?) • Dependency only on a single previous state, • One manifest variable for each state • Conditional random fields: • Used for structured predictions (e.g., speech recognition) • For speech recognition, take into the account the classes of all letters in a word. • Widely used in natural language processing, • More flexible than HMMs, • Challenges with CRF: • Too many parameters to estimate with little data V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 17 / 18
  • 20. Conclusions and future work Summary: • Promising path to explore, • Use of backoff trigrams, plain and backoff dependency triplets, entity count and first message indicator seems useful, Future work: • Additional types of features which look at the context of previous messages (e.g., convergence vs. divergence), • Moving away from SVM, explore other classification methods which are better at explanation • Give associated probabilities for each classification, • Give relative importance of different features. Challenges: • Challenges with message unit of analysis and surface-level features, • Low frequency of resolution messages. V. Kovanovi´c et al. (EDI) Automated Analysis of Discussion Transcripts 31 Aug 2015 18 / 18
  • 22. References I Chang, Chih-Chung and Chih-Jen Lin (2011). “LIBSVM: A library for support vector machines”. In: ACM Transactions on Intelligent Systems and Technology 2 (3), 27:1–27:27. Donnelly, Roisin and John Gardner (2011). “Content analysis of computer conferencing transcripts”. In: Interactive Learning Environments 19.4, pp. 303–315. Garrison, D. Randy, Terry Anderson, and Walter Archer (1999). “Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education”. In: The Internet and Higher Education 2.2–3, pp. 87–105. Henri, France (1992). “Computer Conferencing and Content Analysis”. en. In: Collaborative Learning Through Computer Conferencing, pp. 117–136. Witten, Ian H., Eibe Frank, and Mark A. Hall (2011). Data Mining: Practical Machine Learning Tools and Techniques, Third Edition. 3rd ed. Morgan Kaufmann.