Examples of Socratic Dialogues for Reaching Conclusions on Disinformation

Project Plenary meeting_10 May 2023
Socratic Dialogues
Georgios Petasis, Dora Katsamori
NCSR Demokritos

Agenda
● Theoretical Background
○ The Socratic Method
○ Experiential Learning
○ Critical Thinking Skills
● A methodology for writing dialogues
● Collected Dialogs
● Current “Oversimplification” of the problem
2

The Socratic Method (D2.1)
1. Clarification
What do you mean when you say X?
2. Challenging assumptions
What assumptions are we making here?
3. Evidence and reasoning
Can you provide an example that supports what you are
saying?
4. Alternative viewpoints
Are there alternative viewpoints?
5. Implications and consequences
How would this affect someone?
6. Challenging the question
What do you think was important about that question?
3

Personalised Learning (Not currently used)
● Experience: User comes in touch with a
specific experience – read a news item and
he/she perceives/believes it according to
his gained knowledge, attitude, values, etc.
● Reflective Observation: Ignore the first
impression and try to reflect on it
● Active Experimentation: the user has to try
to detect in text problems
● Concrete Experience: the user is able to
approach the knowledge and to ‘learn how
to learn’
(Johns & Wolking, 2016)
4

Critical Thinking Skills (D2.1 – the only part used)
● Interpretation
● Analysis
● Evaluation
● Inference
● Explanation
● Self-regulation?
5

Socratic Dialogues + Critical Thinking Skills
6

Input from WP5
● WP5 provided a lot of input (per pilot) regarding:
○ Types of disinformation signals
■ emotional text / provocative information / hate speech
■ non cross-checked information / unverified source
■ Fallacies
○ Sample articles (~5 per pilot)
○ Annotated articles with disinformation signals
● Pilot 1: Students – 5 articles
● Pilot 2: NGO’s – 3 articles
● Pilot 3: Migration – 4 articles
7

A set of instructions
● Describes the steps to write a dialogue
8

Approach
Step 1: Annotation
● Each article has been annotated by pilot users according to three types of
problems/reasons:
● emotional text / provocative information / hate speech
● non cross-checked information (x) / unverified source
● fallacies
○ slippery slope
○ implicit assumption
○ red herring
9

Step 2: Dialogue creation
Each expert was asked to select an article and create a Socratic dialogue based
on the following dimensions:
● the level of the critical thinking user (low or media)
● Specific disinformation signals according to the annotated
● The six steps of the Socratic dialogue approach:
○ Clarification
○ Challenging assumptions
○ Evidence and reasoning
○ Alternative viewpoints
○ Implications and consequences
○ Challenging the question
10

Overview of the data collection
● 26 dialogues (Pilot 1: 14, Pilot 2: 2 & pilot 3: 10) from the following partners:
○ 8 dialogues from IPT (2 colleagues involved)
○ 14 dialogues from NCSRD-D (4 colleagues involved)
○ 2 dialogues from VRT (1 colleague involved)
○ 2 dialogues from AHS (1 colleague involved)
11

Next Steps
● Users to validate/expand the set of example dialogues in the living labs
● Try to model them through the rule-based prototype, using the infrastructure
provided by UNIVDUN
● To be enriched by UNINETTUNO according to critical thinking awareness
approach
12

“Oversimplification”
● The dialogues lack important aspects according to D2.1:
○ The issue of trust and critical thinking (VUB)
○ The Social Cognitive Theory, the three-stage model of self-regulatory functioning, and critical
thinking skills
● The SOTA in D2.1 identifies the concerns, and provides a (very) broad
categories of factors
○ We need to elaborate further
■ enumerate factors
● For each factor we need how to detect it, evaluate it and mitigate it
Detection Evaluate Mitigate
Factor Assessment
13

Dialogue example I
EXAMPLE: #3 BOLOGNA, PAKISTANI RAPES GIRL IN ALCOHOL COMA
○ Clarification
User level: Intermediate
Disinformation signal examined: Hate speech
SA: Hi there! I am TITAN/SA(;) your personal AI coach, how can I be of assistance?
U: I came across this article and felt the need to discuss it
SA: Great, let’s take a closer look together then! What made you choose this particular article in the first place?
(…/I don’t know)
U: I found this news disturbing and shocking but also and the language used by the author very emotional
SA: Very well. Are you able to identify the main claim of the article? What does the article discuss?
U: The article discusses the rape of an Italian girl by a Pakistani in a hospital in Bologna. It points out the danger
posed by these individuals and the inability of the state to act accordingly
SA: That’s very interesting! Are you well-acquainted with the topic of the article or is it something relatively new to
you?
14

U: Neither, but I’ve read similar things in the past
SA: Were you able to identify anything unusual about the article then? Can you tell me
what is troubling you?
U: I realized that in certain parts the language used by the author made me feel
uncomfortable
SA: Can you provide me with an example(s) from the text?
U: Some words/phrases like “PD-branded scum” or the fact that the “good-natured lady”
who helped the Pakistani should be denounced as complicit
SA: Are you aware that texts/articles are capable of provoking emotions and feelings
(sometimes even on purpose)? (Yes/No)
U: Yes I am well aware
15

SA: So, being aware of the fact that, sometimes, an article isn’t purely descriptive and its purpose
informative, would you say that this particular text aspires to achieve something more aside from
providing us with information regarding a fact/event? (Yes/No/I don’t know)
U: Yes
SA: What makes you think that?
U: Aside from recounting the story, I feel that the author is trying to alert people on the dangers
posed by illegal immigrants, as well as the inadequacy of the government.
SA: And how do you think that the author achieves that? Can you give me any example(s)? (…/I
don’t know)
U: The use of strong language is clear throughout the text. The author uses words like “scum”, “free
to go and rape another Italian girl”, “shitty state”, etc.
SA: The use of strong and/or emotional language usually results to more evident -if not more biased-
claims. Do you think that the claims made by the author in this article can be considered well
founded? (Yes/No/I don’t know)
16

U: I am not so sure. Some claims do not appear to be well founded, but maybe for a good reason?
The abuse of an unconscious 23-old girl is a very serious thing
SA: Do you think you are able to provide me with an example of a biased claim then? Can you find
any in the text? (Yes/No)
U: Yes. The author claims for no apparent reason that the woman who brought the Pakistani to the
hospital and the “goodists in the government” are complicit to the rape. Also, he appears to think
that hospitals are “hotels for illegal immigrants” and Italian politicians criminals because they
charged the Pakistani with sexual assault but let him “free to go and rape another Italian girl”
SA: Keeping in mind that, sometimes, the use of strong and/or provocative language can be a
product of hate speech, are you, by any chance, able to identify anything offensive about the claims
and assumptions being made in this article? (Yes/No)
U: Yes
SA: Hate speech, unlike other forms of offensive speech/writing, is usually based on prejudices
regarding ethnicity, race, sex, sexual orientation or even religion. Do you feel that it’s possible any
of the assumptions made might have any of the aforementioned characteristics? (Yes/No)
U: Yes
17

SA: And why do you think the author makes these assumptions (if any)? Can you think of any
particular reason? (Yes/No/I don’t know)
U: I don’t know. Maybe he wants to convince us that these people are dangerous and warn us of
the danger they pose
SA: Do you feel that these assumptions are sufficiently justified based on the evidence offered by
the author? (Yes/No/I don’t know)
U: I am not sure. Some of the statements and assumptions seem irrelevant to the story but, on the
other hand, a girl was raped by an illegal immigrant
SA: Sometimes, even if the author provides us with valid information regarding a fact/event, his/her
claims and assumptions might still be offensive or a product of disinformation. Can you find enough
evidence in the text supporting the weaker claims made by its author? (Yes/No)
U: No
18

SA: Do you think that you are somehow able to validate these claims yourself?
(Yes/No)
U: No, I don’t believe I can. I have to admit that most of the claims made by the author
are difficult to validate
SA: Then do you believe that any of these claims might be identified as the result of
hate speech as we previously discussed?
U: Yes definitely
SA: Articles naturally tend to reflect the ideas and beliefs of the author. What is
important though is the ability to distinguish between the personal opinion/beliefs of the
author and the facts/events that are discussed in the text. Do you think you are in a
position to do so? (Yes/No)
U: Yes
19

SA: Regardless of the appeal an explanation might have, it is always important to think/ search for
alternative viewpoints/opinions. Are you, by any chance, able to think of an alternative way(s) of
approaching the subject in question? (Yes/No)
U: Yes
SA: Can you try giving me an example? (…/No)
U: Since the evidence offered regarding some of the claims made by the author is lacking, I think
that the rape of the girl can be perceived as a violent incident triggered by something other than the
ethnic identity of the perpetrator or the responsibility of the “goodists” (whether the woman or the
government)
SA: Naturally an event might be interpreted in many different ways. Therefore, ideally, we always
have to carefully assess the individual strengths and weaknesses of every possible explanation. At
this point, do you believe that perhaps an alternative explanation of the events is more plausible
than the one offered by the author? (Yes/No/I don’t know)
U: Yes, probably
20

SA: So, after carefully considering the assumptions made by the author, can you think of any
consequences that this article might have to a potential reader? (Yes/No)
U: Yes. I think that the article is likely to provoke negative feelings towards immigrants,
especially Pakistanis
SA: Do you believe that the assumptions/claims of the article are consistent with your own
personal experience on the subject (if any)? (Yes/No/I don’t know)
U: Honestly, I don’t believe they are
SA: What would you say that the most important characteristic of the article is then? How
would you describe it?
U: I feel that the article is not simply trying to inform us of a certain event, but also to cultivate
distrust against migrants and the authorities
21

SA: After this short discussion, how do you feel about this article?
U: Keeping in mind the gravity of what transpired, I am reluctant towards the motives and assumptions of
the author
SA: Do you feel that you’ve grasped the reasoning of the author? Are you able to form an alternative
argument? (Yes/No)
U: Yes I think I can
SA: Is there anything you would ideally change regarding the approach of the author? If yes, can you tell
me what? (Yes/No)
U: Yes, I believe that I would never accuse the woman or the government for complicity since there was
hardly any involvement on their part, and also I would avoid using the identity of the perpetrator as an
excuse to demonize immigrants in general
SA: And what do you think of our discussion? Did you find it helpful? (Yes/No)
U: Yes
SA: Is there anything more you would like to discuss with me? (Yes/No)
U: No, thank you
22

Dialogue example II
“The government is spying on you through LED-lighting!!”
○ Clarification
User level: Low
Disinformation signal examined: Logical Fallacy - Appeal to Authority (argumentum ad
auctoritatem)
SA: Hello! Let me introduce myself, my name is TITAN, your AI Intelligent Coach against
Disinformation
U: Hello!
SA: You've selected the article: “The government is spying on you through LED-lighting!!”
U: Yes.
SA: What is the main issue in the article?
U: The article claims that governments and large corporations are using LED lights to monitor
individuals. This is conducted through the Internet of Things.
23

SA: What caught your attention about this article?
U: The topic is extremely interesting and raises an issue of major concern.
SA: What do you already know about the issue?
U: This is the first time I've heard about this issue.
SA: Would you like to learn more about the quality of reasoning in this article?
U: Certainly, yes!
SA: Good reasoning or sound reasoning refers to avoiding logical fallacies or errors in an
argument.
U: That's helpful.
SA: Would you like me to assist you in identifying logical fallacies in this article?
U: Yes, but I don’t know what a logical fallacy is exactly.
SA: A logical fallacy is a flaw in reasoning that makes an argument invalid or unsound. A logical
fallacy can make an argument potentially misleading. [Suggestion: Microlesson on Sound
Reasoning and Logical Fallacies]
U: That’s useful.
24

SA: We can focus on a logical fallacy called Appeal to Authority (argumentum ad
auctoritatem).
U: Fine!
SA: Would you like to learn more about this logical fallacy?
U: Yes. [Suggestion: Microlesson on Appeal to Authority logical fallacy]
SA: Appeal to Authority (argumentum ad auctoritatem) occurs when a claim is supported by
relying on the expertise of an authority figure, or an expert in a field, rather than on sound
reasoning or objective evidence.
U: Can you provide an example?
SA: Here is an example “You should take vitamins A and C because experts in healthcare said they
are the best.” Can you identify the logical fallacy in this claim?
U: Not really, this claim is based on the expertise of healthcare specialists.
SA: Yes, but is there any evidence provided to support the claim?
U: No.
25

SA: This is the reason why this example is a case of an Appeal to Authority fallacy.
U: Can you clarify?
SA: The claim in the example urges us to use vitamins because experts in healthcare
said so, without providing sound reasoning or objective evidence.
U: That’s informative.
SA: Can you identify a similar practice in the article?
U: Yes, I think the following statement “university research all over the world have
shown that LED-lighting is bad for your eyes” [p]
26

SA: What led you to that decision?
U: The expression “university research” appeals to the authority of university researchers
SA: Yes, but can we admit that university research is conducted by experts on a field with
careful scientific research that is published in reputable and reliable academic journals?
U: Yes, we can admit it.
SA: So, why do you suggest that this could be the case of an Appeal to Authority fallacy?
U: There is no evidence provided to support the claim, and without any cited sources.
SA: Brilliant! So, the statement [p] can be called into question.
U: Yes
27

SA: Are there any alternative viewpoints?
U: Perhaps, no.
SA: What about claiming that despite the unspecified evidence of the assertion, the
conclusion might be correct.
U: How is this possible? I'm not sure.
SA: Your question is valid. We need further reasons to doubt the validity of the
statement [p].
U: Yes.
SA: Is there any additional reason to doubt the validity of the statement [p]?
U: I am not sure.
SA: Consider the assertion in statement [p] that university research is globally
applicable, “all over the world”
U: That’s interesting.
28

SA: The phrase is a generalization, a broad expression suggesting that something is globally or
universally true.
U: That’s helpful
SA: Drawing a conclusion based on limited or insufficient evidence can result in another logical
fallacy, known as Hasty Generalization.
U: Could you please explain further? [Suggestion: Microlesson on Logical Fallacies / Hasty
Generalization]
SA: A Hasty Generalization occurs when conclusions are drawn about an entire group or category
based on insufficient or limited evidence, often relying on stereotypes.
U: Can you provide an example?
SA: Consider the claim that “A group of teenagers sabotaged the exhibition. There is no doubt that
teenagers are reckless and vicious”.
U: That’s informative.
SA: So, do you think that statement [p] is also subject to a Hasty Generalization fallacy?
U: Yes.
29

SA: Are there any consequences of accepting statement [p]?
U: I don’t think there are any serious implications.
SA: Could a fallacious claim, which relies on Hasty Generalization and Appeals to
Authority, cause unwarranted anxiety or unnecessary fear in the reader?
U: Yes, if it is taken seriously.
SA: Moreover, do you agree that the statement [p] might bring mistrust to LED-lighting
technologies, and their possible advantages?
U: Yes, I agree.
SA: Would you like to learn more about the implications and consequences of
disinformation?
U: Yes > Go to [Microlesson on Disinformation Implications and Consequences]
U: No > Go to Step 6
30

SA: Was it appropriate to check the article for the Appeal to Authority fallacy?
U: Yes, definitely.
SA: Do you agree that a claim which relies on unsupported authority and broad generalizations, is
not only invalid but also carries potential negative consequences?
U: Yes, indeed.
SA: So, considering the Appeal to Authority fallacy, the claim should be doubted, and the article is
in question.
U: Yes.
SA: We can also assert that the argument is not valid, as far as the evidence provided is not
sufficient to support the claim.
U: Correct.
31

SA: What would be a recommended course of action once an Appeal to Authority fallacy
has been identified?
U: I am not sure.
SA: I would suggest examining the validity of statement [p], by finding reliable evidence and
unbiased sources.
U: That’s a good suggestion.
SA: Would you recommend any additional evaluation of the article?
U: Perhaps, the article should be checked for more logical fallacies.
SA: Would you like further assistance?
U: Yes [= Go to Step 3]
U: No
SA: Thank you for using TITAN!
32

Challenges of the dialogue creation
● different understanding and approach from the experts/creators
● lack of a specific technique for creation
● lack of alternatives in answers
● difficulty in linking questions between 6 steps
● focus on the discussion around the content of each article, without highlighting the
educational/teaching part in order to increase the critical thinking level of the users
● Connection of experiential learning
33

Thank You!
This project has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101070658. This
presentation reflects only the view of the authors, and the European Commission is not responsible for any use that may be made of the information it contains.

Examples of Socratic Dialogues for Reaching Conclusions on Disinformation

More Related Content

Similar to Examples of Socratic Dialogues for Reaching Conclusions on Disinformation

More from TitanEurope1

Recently uploaded

Examples of Socratic Dialogues for Reaching Conclusions on Disinformation