A joint keynote with Heather O'Brien at the Learning Analytics Summer Institute (LASI) 2019. In here we explore the concept of learner- and user- engagement as relevant for the field of learning analytics.
JISC RSC London Workshop - Learner analyticsJames Ballard
Introduction to learning analytics and approaches to learner engagement to raise awareness and set the seen for upcoming projects and advice for supported learning providers.
A joint keynote with Heather O'Brien at the Learning Analytics Summer Institute (LASI) 2019. In here we explore the concept of learner- and user- engagement as relevant for the field of learning analytics.
JISC RSC London Workshop - Learner analyticsJames Ballard
Introduction to learning analytics and approaches to learner engagement to raise awareness and set the seen for upcoming projects and advice for supported learning providers.
Paper presented at the 1st International Conference on Technology and Innovation in Learning, Teaching and Education (TECH-EDU 2018), June 20-22, 2018, Thessaloniki, Greece.
Read more at: http://bit.ly/techedu4
Paper presented at the 1st International Conference on Technology and Innovation in Learning, Teaching and Education (TECH-EDU 2018), June 20-22, 2018, Thessaloniki, Greece.
Read more at: http://bit.ly/techedu4
by Kan Min-Yen, Deputy Director (Research) of
NUS Institute for the Application of Learning Sciences and Education Technology
5th IBC EduCon, Singapore, 28 Sep 2017
Taking evidence-based professional learning conversations online: Implicatio...mddhani
Presented in one of the parallel sessions during the 15th International Conference on Education 2010 at Universiti Brunei Darussalam.
Presenter/courtesy of Michael Moroney, Lecturer, Universiti Brunei Darussalam.
Presented in one of the parallel sessions during the 15th International Conference on Education 2010 at Universiti Brunei Darussalam.
Presenter/courtesy of Michael Moroney, Lecturer, Universiti Brunei Darussalam.
Social and Cognitive Presence in Virtual Learning Environments Terry Anderson
Reviews and speculates on further development of the Community of Inquiry model (communitiesofinquiry.com) developed in Alberta by Randy Garrison, Terry Anderson, Walter Archer and Liam Rourke. This project developed theory and tools to measure teaching, cognitive and social presence in online environments
This is a brief presentation on the subject of Discourse Analysis as a Research Method in Education. This was created by Apostolos Koutropoulos and Rosemarri Klamn for EDDE 802, assignment 2.
This presentation is based on a pilot study and dissertation on reciprocal teaching in a community college course for higher levels of learning using discussion forums.
Education and Technology Partnerships as Intercultural Communities: An Ethnog...CITE
CITERS2014 - Learning without Limits?
http://citers2014.cite.hku.hk/program-overview/keynote-green/
13 June 2014 (Friday)
09:10 – 10:00
Keynote 1: Education and Technology Partnerships as Intercultural Communities: An Ethnographic Perspective
Speaker: Professor Judith GREEN (Department of Education, University of California, Santa Barbara)
Chair: Dr. Susan BRIDGES (Associate Professor, Faculty of Education, HKU)
What Does Conversational Information Access Exactly Mean and How to Evaluate It?krisztianbalog
This talk discusses a set of specific tasks and scenarios related to information access within the vast space that is casually referred to as conversational AI. While most of these problems have been identified in the literature for quite some time now, progress has been limited. Apart from the inherently challenging nature of these problems, the lack of progress, in large part, can be attributed to the shortage of appropriate evaluation methodology and resources. This talk presents some recent work towards filling this gap.
In one line of research, we investigate the presentation of tabular search results in a conversational setting. Instead of generating a static summary of a result table, we complement brief summaries with clues that invite further exploration, thereby taking advantage of the conversational paradigm. One of the main contributions of this study is the development of a test collection using crowdsourcing.
Another line of work focuses on large-scale evaluation of conversational recommender systems via simulated users. Building on the well-established agenda-based simulation framework from dialogue systems research, we develop interaction and preference models specific to the item recommendation scenario. For evaluation, we compare three existing conversational movie recommender systems with both real and simulated users, and observe high correlation between the two means of evaluation.
This talk has been given at the CIIR talk series at the University of Massachusetts Amherst in Jan 2021 as well as at the IR seminar series at the University of Glasgow in March 2021.
Designing Social Interactions in a Teachable Agentdiannepatricia
Erin Walker, Arizona State University presents "Designing Social Interactions in a Teachable Agent" as part of the Cognitive Systems Institute Speaker Series on 9/22/16.
Multiple Methods and Techniques in Analyzing Computer-Supported Collaborative...CITE
5 March 2010 (Friday) | 09:00 - 12:30 | http://citers2010.cite.hku.hk/abstract/69 | Dr. Jingyan LU, Research Assistant Professor, Faculty of Education, HKU
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
The Roman Empire A Historical Colossus.pdfkaushalkr1407
The Roman Empire, a vast and enduring power, stands as one of history's most remarkable civilizations, leaving an indelible imprint on the world. It emerged from the Roman Republic, transitioning into an imperial powerhouse under the leadership of Augustus Caesar in 27 BCE. This transformation marked the beginning of an era defined by unprecedented territorial expansion, architectural marvels, and profound cultural influence.
The empire's roots lie in the city of Rome, founded, according to legend, by Romulus in 753 BCE. Over centuries, Rome evolved from a small settlement to a formidable republic, characterized by a complex political system with elected officials and checks on power. However, internal strife, class conflicts, and military ambitions paved the way for the end of the Republic. Julius Caesar’s dictatorship and subsequent assassination in 44 BCE created a power vacuum, leading to a civil war. Octavian, later Augustus, emerged victorious, heralding the Roman Empire’s birth.
Under Augustus, the empire experienced the Pax Romana, a 200-year period of relative peace and stability. Augustus reformed the military, established efficient administrative systems, and initiated grand construction projects. The empire's borders expanded, encompassing territories from Britain to Egypt and from Spain to the Euphrates. Roman legions, renowned for their discipline and engineering prowess, secured and maintained these vast territories, building roads, fortifications, and cities that facilitated control and integration.
The Roman Empire’s society was hierarchical, with a rigid class system. At the top were the patricians, wealthy elites who held significant political power. Below them were the plebeians, free citizens with limited political influence, and the vast numbers of slaves who formed the backbone of the economy. The family unit was central, governed by the paterfamilias, the male head who held absolute authority.
Culturally, the Romans were eclectic, absorbing and adapting elements from the civilizations they encountered, particularly the Greeks. Roman art, literature, and philosophy reflected this synthesis, creating a rich cultural tapestry. Latin, the Roman language, became the lingua franca of the Western world, influencing numerous modern languages.
Roman architecture and engineering achievements were monumental. They perfected the arch, vault, and dome, constructing enduring structures like the Colosseum, Pantheon, and aqueducts. These engineering marvels not only showcased Roman ingenuity but also served practical purposes, from public entertainment to water supply.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
2. What do we want to enable?
• Project and Problem based learning
• Collaborative reflection and Collaborative Problem Solving
• Community building and social support
– Reducing attrition in MOOCsGateways to enduring communities of practice
– Bridging learning and practice
2
4. Conversational Agent Based Support in
Computer Supported Collaborative Learning
Students learn 1.24 s.d. more when working with a partner and automated support than
students working alone (Kumar et al., 2007)
5. Technology Support for Collaborative
Learning
Automatic
Analysis
Of
Conversation
Conversational
Interventions
Positive
Learning
Outcomes
6. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
7. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
12. Data Analytics Pipeline
• Browse data in DiscourseDB
• Import/Export data
• View, manipulate, create Annotations
• http://brat.nlplab.org
• Use annotations on DiscourseDB data to
train models.
• Use models to annotate DiscourseDB data
• http://ankara.lti.cs.cmu.edu/side
12
14. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
15. Automated Essay Scoring as an Example
• Earliest approaches to automated essay scoring (Page,
1966)
• Work towards greater validity (Shermis & Burstein, 2003)
• Work towards triggering feedback from assessment
(Wade-Stein & Kintch, 2004)
• Competition across industry and academia (Shermis &
Hammer, 2012)
– Free off-the-shelf approach just as good as industry options
– LightSIDE (Mayfield & Rosé, 2013)
15
16. What did we learn?
• Simple features can make accurate predictions
– Word features, sentence length, rare word count, etc.
– They function as proxies for more meaningful features
• E.g., longer sentences may have more elaboration, so
sentence length might correlate with amount of elaboraton
• Models trained with simple features are
problematic
– They don’t transfer well between contexts
– They are not useful for instructions
21. Language
Technologies
Theoretical Framework
• Basic concepts of power and social
distance explain social processes
operating in interactions
• Social processes are reflected
through patterns of language
variation
• If we understand this connection,
we can model language more
effectively
• Models that embody these
structures will be able to predict
social processes from interaction
data
Psychology
Sociolinguistics
27. Power, Relationships, and Transactivity
1963 1983 1986 1993
Piaget Berkowitz
& Gibbs
Kruger &
Tomasello
Azmitia
& Montgomery
Power,
Cognitive Conflict,
And Learning
Socio-
Cognitive
Conflict and
Transactivity
Power Balance
And
Transactivity
Friendship,
Transactivity,
And Learning
28. From the Social to the Cognitive
Findings from
Sociolinguistics
DBM
Model
Findings from
Developmental
Psychology
Model connecting
speech style
accommodation and
Transactivity
Jain et al., 2012 Gweon et al., 2012
Reflecting
Perspectives
And
Relationships
Reflecting
Evidence of
Consensus Building
29. Computational Modeling the Connection
between Cognitive and Social Factors
A speaker’s style depends both
on mutual accommodation and
the partner’s style in the
previous turn.
Accommodation states signal
that if accommodation is
happening, it is likely to persist
29
Measured accommodation through DBM predicts Prevalence of Transactivity.
Gweon, G., Jain, M., Mc Donough, J., Raj, B., Rosé, C. P. (2013). Measuring Prevalence of Other-Oriented Transactive Contributions Using an
Automated Measure of Speech Style Accommodation, International Journal of Computer Supported Collaborative Learning 8(2), pp 245-265.
31. • Findings
– Moderating effect on learning (Joshi & Rosé, 2007; Russell, 2005; Kruger &
Tomasello, 1986; Teasley, 1995)
– Moderating effect on knowledge sharing in working groups (Gweon et al., 2011)
• Computational Work
– Can be automatically detected in:
• Threaded group discussions (Kappa .69) (Rosé et al., 2008)
• Transcribed classroom discussions (Kappa .69) (Ai et al., 2010)
• Speech from dyadic discussions (R = .37) (Gweon et al., 2012)
– Predictable from a measure of speech style accommodation computed by an
unsupervised Dynamic Bayesian Network (Jain et al., 2012)
Transactivity (Berkowitz & Gibbs, 1983)
33. • Findings
– Correlational analysis: Strong correlation between displayed
openness of group members and articulation of reasoning (R =
.72) (Dyke et al., in press)
– Intervention study: Causal effect on propensity to articulate ideas
in group chats (effect size .6 standard deviations) (Kumar et al.,
2011)
• Mediating effect of idea contribution on learning in scientific
inquiry (Wang et al., 2011)
Engagement (Martin & White, 2005)
35. Analysis of Authoritativess
Water pipe analogy:
Water = Knowledge or Action
Source = Authoritative speaker
Sink = Non-authoritative Speaker
Authoritativeness Ratio = Source Actions
Actions
36. The Negotiation Framework
(Martin & Rose, 2003)
Source
orSink?
Prim
ary
Secondary
Type
ofContent?
Know
ledge
Action
K2
requesting knowledge,
information, opinions, or facts
K1
giving knowledge, information,
opinions, or facts
A2
Instructing, suggesting, or
requesting non-verbal action
A1
Narrating or performing your
own non-verbal actionAdditionally…
ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
K1 + A1
K1 + K2 + A1 + A2
Authoritativeness:
37. Where did the Negotiation framework
come from?
Systemic Functional Linguistics: The study of how people talk to
each other (Martin, 2003)
Functional, rather than generative, approach to describing
language.
Negotiation framework has been used in sociolinguistic literature
since the 1980s
(e.g. Berry, 1981; Veel, 1999; Martin, 2008)
38. The Negotiation Framework
(Martin & Rose, 2003)
Source
orSink?
Prim
ary
Secondary
Type
ofContent?
Know
ledge
Action
K2
requesting knowledge,
information, opinions, or facts
K1
giving knowledge, information,
opinions, or facts
A2
Instructing, suggesting, or
requesting non-verbal action
A1
Narrating or performing your
own non-verbal action
Additionally…
ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
39. The Negotiation Framework
(Martin & Rose, 2003)
Source
orSink?
Prim
ary
Secondary
Type
ofContent?
Know
ledge
Action
K2
requesting knowledge,
information, opinions, or facts
K1
giving knowledge, information,
opinions, or facts
A2
Instructing, suggesting, or
requesting non-verbal action
A1
Narrating or performing your
own non-verbal action
42. The Negotiation Framework
(Martin & Rose, 2003)
Source
orSink?
Prim
ary
Secondary
Type
ofContent?
Know
ledge
Action
K2
requesting knowledge,
information, opinions, or facts
K1
giving knowledge, information,
opinions, or facts
A2
Instructing, suggesting, or
requesting non-verbal action
A1
Narrating or performing your
own non-verbal action
Additionally…
ch (direct challenge to previous utterance)
o (all other moves, backchannels, etc.)
45. • Findings
– Authoritativeness measures display how students respond to aggressive
behavior in groups (Howley et al., in press)
– Authoritativeness predicts learning (R = .64) and self-efficacy (R = .35) (Howley
et al., 2011)
– Authoritativeness predicts trust in doctor-patient interactions (R values
between .25 and .35) (Mayfield et al., under review)
• Computational Work
– Detectable in collaborative learning chat logs (R = .86)
– Detectable in transcribed dyadic discussions in a knowledge sharing task (R =
.95) (Mayfield & Rosé, 2011)
– Detectable in transcribed doctor-patient interactions (R = .96) (Mayfield et al.,
under review)
Authoritativeness (Martin & Rose, 2003)
46. Example: MathTalk
• Personalized Agent
condition vs
Control condition
• 30 6th
graders
– Randomly assigned to
pairs, conditions
• Procedure
46
Social Dialogue Agent Study (Kumar et al, 2007)
Day 1 Day 2
Lab session Lab session
Pretest Quiz Posttest Quiz,
Questionnaire
48. Main Results:
Advantage for Social Condition
• Significant increase in perception of amount of
help given and received
– Significant increase in amount of help given per problem
(Gweon et al., 2007)
– Students marginally more likely to complete a step on their own
after receiving help (Cui et al., 2009)
• Marginally higher learning gains
• But why?
48
[Kumar et al, 2007]
49. Understanding the Effect of Social Climate
on Positioning and Risk Taking
• Coded chat logs for instances of aggressive
behavior
– Pushy behavior
– Insults
• Coded for Negotiation (especially K1 and K2)
– Based on counts of K1 and K2, computed an authoritativeness
score for each student per lab day
• K1/[K1 + K2]
– Computed a Shift score per student
• Residual from linear regression predicting Day 2
authoritativeness from Day 1 authoritativeness
– Binary Shift variable (within pair, which student shifted up to a
more authoritative stance versus shifted down)
50. Aggressive Behavior
• Significantly more aggressive behavior
in Control condition
– F(1,56) = 8.93, p < .005 **, effect size .63σ
• Significantly more aggressive behavior
on Day 2
– F(1,56) = 15.61, p < .0005 **, effect size .87σ
– Significant interaction with Condition
• F(1,56) = 6.06, p < .05 **
• Only significant increase in aggressive
behavior on Day 2 in the Control
condition
• In each pair, identified student with
higher amount of aggressive behavior
on Day 2 as “the bully” for further
analysis
51. Authoritativeness and Shift
• Significant difference in
Authoritativeness of Bullies
and Non-Bullies in Control
condition on Day 2
– F(1,23) = 5.92, p < .05**
• Visible Shift only in Control
Condition
– F(1,23) = 5.28, p < .05**, effect size .15σ
– Bullies in Control condition shifted to
more authoritative stance
– Non-bullies in Control condition shifted
to less authoritative stance
51
AuthoritativenessShift
52. Learning
• No significant main effect of Aggressive
behavior on learning
– Bullied students in Control condition learned
significantly less than Social Condition students
– Recall that students respond differently to help in
Control condition
• Significant interaction between Shift and
Condition on Learning: F(1,20) = 7.91, p =
.01**
– Opposite trend in Social Condition
– Significant correlation between amount of shift and
learning only within Control condition
• Shifting down was associated with less
learning
BullyingShift
53. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
55. 55
Measuring Cognitive Engagement
• Displays effort in interpreting,
reflecting on, and reasoning
about course material
• Used a publically available
Abstractness dictionary
(Turney et al., 2011)
56. 56
Measuring Motivation
• 514 Posts labeled by Mechanical
Turk, on a 1-7 likert scale
(Extremely Unmotivated …
Extremely Motivated)
• Each example rated by 6 Mturkers
(Intraclass correlation = .74)
• Classifier binary classifier (median
split) trained using LibLinear with
L2 regularization (72.3% accurate)
59. Confusion Detection
Goal - Build models to
automatically identify the level of
confusion expressed in students’
posts
1. Create the Human-Coded Dataset:
MTurk
2. Feature Space Design: click
behavior, domain-specific content
words, linguistic features
3. Train classifier over annotated data
4. Apply the classifier to all posts in a
course
60. Human-Coded Dataset: MTurk
• Amazon Mechanical Turk Labeling
– For each post ($0.06), judge the level of confusion
contained in the message in a 1-4 Likert scale
– Ranging from ‘No Confusion’, ‘Slightly Confused’,
‘Moderately Confused’ and ‘Seriously Confused’
• Intra-Class Correlation
– 0.745 (Algebra), 0.801 (Microeconomics)
• Agreement between MTurkers’ average ratings
and expert’s labeling
– 0.86 (Algebra), 0.80 (Micro)
61. Feature Space Design
• Click patterns reveal sequences of activities associated with confusion
– Patterns consist of taking quizzes (quiz), watching lectures (lecture),participating in
forums (forum), and viewing other course materials (course)
• Language Features from Forum posts
• Linguistic Indicators
– Pronouns : “I, we, you, she/he”;
– Sentiment : negative affect
– Grammar: negation, disfluencies, adverbs
• Question Indicators
– Question Markers ‘?’
– Whether sentences begin with confusion related expression
– Whether sentences start with a modal verb/question word
69. ● Reflection activities offered as
optional supplements at the
end of each unit
● If students click to enter the
activity, they may be required
to do it individually or
collaboratively
○ Out of 14,000 clicks to
enter the reflection
activities, 25% included an
additional student
Survival Analysis from Medicinal Chemistry MOOC
70. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
71. Technology Support for Collaborative
Learning
Automatic
Analysis
Of
Conversation
Conversational
Interventions
Positive
Learning
Outcomes
77. 77
Machine Learning is not just Algorithms!
Data Collection Anonymization Sampling Cleaning
ReformattingFeature Space
Representation
Feature SelectionModeling
80. Nguyen, D., Dogruöz, A. S., Rosé, C. P., de Jong, F.
(2016). Computational Sociolinguistics: An Emerging
Area for Language Technologies, Computational
Linguistics, Vol. 42, No. 3: 537–593.
80
81. Outline
• Process Overview
• Theoretical Framing for Discourse Analysis
• Motivating design through corpus studies
• Zooming in on Text Analytic Process and
Technology Support
• Tools and Resources
82. Resources
DANCE Discussion Forum is
compatible with Open edX
Includes hooks for
interventions like Social
Recommendation and
Discussion Scaffolding
83. Resources
LightSIDE
Text mining tool bench
Over 10,000 users have downloaded LightSIDE
Automated collaborative process analysis
Automated writing assessment/feedback
generation
Social Recommendation
deployed so far in one MOOC to support help
exchange
84. Conclusion
• Join us: Open Source Resources
– Let me know if you would like to collaborate
87. Entailment: Wikipedia Definition
• In semantics, entailments depend on the
"dictionary definition" of the words in question.
• To judge whether an entailment is true, one
can ask, "Could it ever be the case that B isn't
true while A is true?"
88. Entailment: Example
• Example from M. Lynne
Murphy's Lexical Meaning
• "If it is a shoe, then it is
made to be worn on a foot."
89. Entailment Dataset
• Stanford Natural Language Inference Corpus, Bowman et al. 2015.
• Collection of 570,000 English sentence pairs labeled for balanced
classification of entailment, contradiction, and neutral.
• Examples were generated by humans in response to sentences
describing pictures from Flickr
• Example:
– Sentence1: “A soccer game with multiple males playing.”
– Sentence2: “Some men are playing a sport.”
92. Step 1: Attend
• For each pair of words in the two posts, determine some
attention score via 2 layer dense feed-forward neural
network.
• For each word in each post, average all the attention scores
with relation to the other post.
• What you get:
– Information that indicates how important each word in a given
post is with respect to the other post.
94. Step 2: Compare
• Using the representation from the attention step along
with the corresponding vectorized input post, run
though a 2 layer dense feedforward neural network.
• What you get:
– Two sets of vectors for that contain information comparing
the posts with respect to each other.
96. Step 3: Aggregate
• Sum each set of comparison vectors into two
one dimensional vectors.
• Each of these vectors is a representation of a
given post in relation to the other.
98. Step 4: Classify
• With the resulting vectors from the aggregation
step, we concatenate them and run them
through another 2 layer dense feedforward
neural network with cross-entropy loss to
classify the data.
99. Experiment 1
1. Train Entailment task first
2. Use trained weights as initialization for
Transactivity task
3. Train on Transactivity task
100. Transactivity Dataset
• Discussion data from online forum where students offered feedback to one another
on their proposals for city power plans
• 476 human annotated posts.
• Example:
– Sentence 1:
“But if the energy is saving them some money it could go towards the batteries. Whats
frustrating is that it doesn't really give us information regarding the costs of generating
electricity currently.”
– Sentence 2:
“But those batteries add even more cost, and for a city concerned with cost, that would be a
problem. Plus, without the batteries, it's not very reliable, and that's also a problem for a
touristry driven economy.”
101. Results, Part 1
Model Accuracy Cohen’s Kappa
Logistic Regression with unigrams 0.795 0.510
Logistic Regression with
embeddings
0.626 0.182
Our model 0.848 0.542
102. Experiment 2
• Transactivity prediction with in domain data vs. out of
domain data
• Train the model as in experiment 1, however on each cross
validation fold, evaluate the model on out of domain
annotated transactivity data.
• Note that there is no point in which the model is trained on
the out of domain data.
103. Out of Domain Transactivity Dataset
• 57 human annotated transacts from an
Massive Open Online Course (MOOC) in which
students were asked to design their own
superheroes and provide feedback on other
students’ designs.
104. Results, Part 2
Model Accuracy (in |
out)
Cohen’s Kappa (in
| out)
Logistic Regression with
unigrams
0.795 | 0.667 0.510 | 0.376
Logistic Regression with
embeddings
0.626 | 0.635 0.182 | 0.195
Our model 0.848 | 0.824 0.542 | 0.586
105. Machine Learning for Negotiation
Results given are from 20-fold
leave-one-conversation-out cross validation
All improvements between models are
significant (p < .01)
Tools used:
•LightSIDE (Mayfield and Rosé, 2010) for feature extraction
•SVMlight
(Joachims, 1999) for machine learning
•Learning-Based Java (Rizzolo and Roth, 2010) for ILP inference
106. SFL Researchers have found that,
in general… (Martin and Rose, 2003)
1. You don’t request information or action after
it’s been given.
2. Knowledge and action don’t mix.
3. You don’t respond to the same request
twice.
4. You don’t respond to your own requests.
107. Integer Linear Programming allows
us to require our classifier to fit to
these patterns (quickly)!
When a prediction would break a
rule, force it to start a new sequence
or back off to a less likely label.
109. Improvements to our feature space
• Lexical and part-of-speech bigrams
• Cosine similarity to previous line
• Predicted label of previous line
• Separate segmentation models for short
(<4 words) and long lines
• Segmentation features for speaker shift