SlideShare a Scribd company logo
A Family of Reactive-Cognitive Architectures based on Natural
Language Processing, as Decision-Making Helpers for IoT, in the
Closed- or Open-World Assumption
Ph.D. Thesis Defense
Candidate: Carmelo Fabio Longo
Advisor: Corrado Santoro
February 17, 2022
Motivation and Background
OS-1 "Samantha" from "Her" HAL 9000 from "2001: Space Odissey"
• High deductive capabilities
• Physical interaction with the environment
• Meta-Reasoning in conceptual spaces for the task of decision-
making
• Session Awareness
• History Awareness
• Consciousness
• High deductive capabilities
• Physical interaction with the environment
• Meta-Reasoning in conceptual spaces for the task of decision-
making
• Session Awareness
• History Awareness
• Consciousness
[1] P. Thagard. “Critical thinking and informal logic: Neuropsychological perspectives.” In: Informal logic, 51
(2011), pp. 152–170. doi: https://doi.org/10.22329/il.v31i3.3398.
GAP
Fast environments with
difficult-to-predict dynamics of
the world
High-Level Cognition
Behaviours (Language,
Complex Planning, Deductive
Reasoning, etc.)
• Language
• Complex Planning
• Deductive Reasoning
• Differential equations
• Statistics
• Signal Processing
Meta-Reasoning
Reasoning in Conceptual Spaces
C.A.S.P.A.R.
Cognitive Architecture System Planned And Reactive
[1] Carmelo Fabio Longo, Francesco Longo, Corrado Santoro “CASPAR: towards Decision Making Helpers Agents for IoT, based on Natural Language
and First Order Logic Reasoning”, in Engineering Application of Artificial Intelligence, Elsevier, 2021.
• Language
• Complex Planning
• Deductive Reasoning
• Differential equations
• Statistics
• Signal Processing
Meta-Reasoning
CASPAR: Cognitive Architecture System Planned and Reactive
• High deductive capabilities
• Physical interaction with the environment
• Meta-Reasoning in conceptual spaces for the task of decision-making
• Session Awareness
• History Awareness
• Consciousness
A pipeline of five modules with the task of taking a sound stream in
natural language and translating it in a neo-davidsonian First Order
Logic (FOL) expression inheriting the shape from the event-based
formal representation of Davidson[1].
[1] D. Davidson, “The logical form of action sentences,” in The logic of decision and action, p. 81–95, University of Pittsburg Press,
1967.
Brutus stabbed suddenly Caesar in the agora
∃e stabbed(e, Brutus, Caesar) ⋀ suddenly(e) ⋀ in(e, agora)
e = davidsonian variable
The MST Builder, has the purpose to build a novel semantic structure
defined as Macro Semantic Table (MST). It summarizes in a canonical
shape all the semantic features in a sentence, by leveraging a step-by-
step dependencies analysis, in order to derive FOL expressions.
MST(u) = {ACTIONS, VARLIST, PREPS, BINDS, COMPS, CONDS}
ACTIONS = [(label𝑘, e𝑘, x𝑖 , x𝑗),...]
VARLIST = [(x1, label1),...(x𝑛, label𝑛)]
PREPS = [(label𝑗, (e𝑘 | x𝑖), x𝑗),...]
BINDS = [(label𝑖, label𝑗),...]
COMPS = [(label𝑖, label𝑗),...]
CONDS = [e1, e2,...]
where u = utterance in natural language
ACTIONS = [(shines:VBZ, e1, x1 , x2), (is:VBZ, e2, x3 , x4)]
VARLIST = [(x1,sun:NN), (x2, ?), (𝑒1, suddenly:RB), (x3, Robert:NN), (x4, happy:JJ), (e1, strongly:RB)]
PREPS = []
BINDS = []
COMPS = []
CONDS = [e1]
When the sun shines strongly, Robert is happy
shines:VBZ(e1, x1, __) ⋀ sun:NN(x1)⋀ strongly:RB(e1) ⟹ is:VBZ(e2, x3 , x4) ⋀ Robert:NN(x3) ⋀
happy:JJ (x4)
MST =
FOL Builder
• Managed by the BDI Framework PHIDIAS
• The Speech-To-Text (STT) Front-End: Beliefs generator
• Each Sensor Instance: it asserts specific beliefs related to
sensors, such as:
• Microphones
• Temperature sensors
• Video Capturing
• Etc.
Set the cooler at 27 degrees in the bedroom
set:VB(d1, __, x1) ⋀ cooler:NN(x1) ⋀ at:IN(d1, x2) ⋀ 27:CD(x2) ⋀
degree:NNS(x2) ⋀ in:IN(x2, x3) ⋀ bedroom:NN(x3)
INTENT(set, cooler, bedroom, (at 26 degree))
Translation Service
Direct Command Parser
Turn off the lights in the living room, when the temperature
is 25 and the time is 12
be:VBZ(d2, x3, x4) ⋀ be:VBZ(d3, x5, x6) ⋀ temperature:NN(x3) ⋀
25:CD(x4) ⋀ time:NN(x5) ⋀ 12:CD(x6) ⟹ turn:VB(d1, __, x1) ⋀ off:RP(d1)
⋀ light:NNS(x1) ⋀ in:IN(d1, x2) ⋀ living:NN(x2) ⋀ room:NN(x2)
COND(337538, be, temperature, 25), COND(337538, be, time, 12)
ROUTINE(337538, turn, light, living room, off)
Translation Service
Routines Command Parser
COND(337538, be, temperature, 25), COND(337538, be, time, 12)
ROUTINE(337538, turn, light, living room, off)
Asserted by Sensor Instances:
+SENSOR(be, temperature, 25)
+SENSOR(be, time, 12)
INTENT(turn, light, living room, off)
shines:VBZ(e1, x1, __) ⋀ sun:NN(x1) ⋀ strongly:RB(e1) ⟹ is:VBZ(e2, x3 , x4)
⋀ Robert:NN(x3) ⋀ happy:JJ(x4)
Definite clauses Builder
shine:VBZ(sun:NN(x1), __) ⟹ be:VBZ(Robert:NNP(x3), happy:JJ(x4))
When the sun shines strongly, Robert is happy
Translation Service
• He likes to eat a bass
• He likes to play the bass
PROBLEM: where “bass” is intended as fish and where as musical
instrument?
INSIGHT: in a common sense words embedding, the bag of
words (context) used for the gloss/examples related to a
synset comprising a lemma, should be likely vectorial
closer (among all synsets comprising that lemma) to
another bag of words making effective usage of such a
lemma (in a similar context).
Unsupervised [1] naive “bag of words” strategy, taking in account of the higher word2vect similarity (sim)
for each lemma l of the sentence S (depending of the richness of the domain):
[1] Navigli, R., 2009. Word sense disambiguation: A survey. ACM Comput. Surv. 41 (2), 10.
http://dx.doi.org/10.1145/1459352.1459355.
• He likes to eat a bass
Like.v.05:VBZ_Feed.v.06:VB(He:PRP(x1), Sea_bass.n.01:NN(x2))
• He likes to play the bass
Like.v.05:VBZ_Play.v.18:VB(He:PRP(x1), Bass.n.07:NN(x2))
Where the glosses are:
Sea_bass.n.01: the lean flesh of a saltwater fish of the family Serranidae
Bass.n.07: the member with the lowest range of a family of musical instruments
# turn off the light in the kitchen
+INTENT(X, "light", "kitchen", T) / lemma_in_syn(X, "change_state.v.01") >>
[exec_cmd("change_state.v.01", "light", "kitchen", T)
# turn off the light in the garage
+INTENT(X, "alarm", "garage", T) / (lemma_in_syn(X, "change_state.v.01") &
eval_cls("At_IN(Be_VBP(Person_NN(x1), __), Home_NN(x2))")) >>
[exec("change_state.v.01", "alarm", "garage", T)]
where:
• X = Verb, T = parameters
• eval_cls(C) is an Active belief = True if C can be deducted from Clauses KB, False otherwise
• At_IN(Be_VBP(Person_NN(x1), __), Home_NN(x2)) represents the sentence: A person is
at home
• Nono is an hostile nation
• Colonel West is American
• missiles are weapons
• Colonel West sells missiles to Nono
• When an American sells weapons to a hostile nation, that American is a criminal
• Be(Nono(x1), Hostile(Nation(x2)))
• Be(Colonel_West(x1), American(x2))
• Be(Missile(x1), Weapon(x2))
• To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3))
• To(Sell(American(x1), Weapon(x2)), Hostile(Nation(x3)) ⟹ Be(American(x4), Criminal(x5))
Clauses KB
Question: Colonel West is a criminal?
Query: Be(Colonel_West(x1), Criminal(x2)) ? FALSE!
[1] P.N. Stuart J. Russell, Artificial Intelligence: A Modern Approach, Pearson, 2010, Chapter 9.3.
Roy_NNP(y) ⟹ man_NN(y) Assignment Rule
Roy_NNP(y) ⟹ man_NN(y)
talk_VBZ(Roy_NNP(x1), woman_NN(x2))
Clauses KB:
talk_VBZ(Roy_NNP(x1), woman_NN(x2)) ⟹ talk_VBZ(man_NN(x1), woman_NN(x2))
Assert implicitly
be:VBZ(e1, x1, x2) ∧ Roy:NNP(x1) ∧ man:NN(x2)
Roy is a man
If the lemma be is disambiguated with be.v.01, which is defined by its gloss as: have the quality of being
[something], we can assert implicitly the following clause:
(Roy talk with a woman)
(Roy talk with a woman ⟹ A man talk with a woman)
Having a simple KB as it follows:
𝑃1(𝐺1(x1)) ⋀ 𝑃2(𝐺2(x2)) ⟹ 𝑃3(𝐹3(x3))
𝑃1(𝐹1(x1))
𝑃2(𝐹2(x2))
Clause Conceptual Generalization of KB is:
𝑃1(𝐺1 1(x1)) ⋀ 𝑃2(𝐺2(x2)) ⟹ 𝐹3(x3)
𝐹1(x1)
𝐹2(x2)
𝑃1, 𝑃2, 𝑃3 can be considered as modifiers of respectively 𝐹1, 𝐹2, 𝐹3.
The body (or left hand-side) of the implication is unchanged to hold the quality of the axiom.
When the sun shines hard, Barbara drinks slowly a fresh lemonade
Hard(Shine(Sun(x1), __)) ⟹ Drink(Barbara(x3), Lemonade(x4))
Hard(Shine(Sun(x1), __)) ⟹ Slowly(Drink(Barbara(x3), Lemonade(x4)))
Hard(Shine(Sun(x1), __)) ⟹ Drink(Barbara(x3), Fresh(Lemonade(x4)))
Hard(Shine(Sun(x1), __)) ⟹ Slowly(Drink(Barbara(x3), Fresh(Lemonade(x4))))
Generalizations
Be(Nono(x1), Nation(x2))
Be(Nono(x1), Hostile(Nation(x2)))
Nono(x) ==> Nation(x)
Nono(x) ==> Hostile(Nation(x))
Be(Colonel_West(x1), American(x2))
Colonel_West(x) ==> American(x) New Query: Be(American(x4), Criminal(x5)) ? (NESTED REASONING)
Be(Missile(x1), Weapon(x2))
Missile(x) ==> Weapon(x)
Sell(Colonel_West(x1), Missile(x2)) ==> Sell(American(v_0), Missile(x2))
Sell(Colonel_West(x1), Missile(x2)) ==> Sell(American(x), Weapon(v_1))
Sell(Colonel_West(x1), Missile(x2)) ==> Sell(Colonel_West(x1), Weapon(v_2))
Sell(Colonel_West(x1), Missile(x2))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(x1), Missile(x2)), Nation(v_4))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_5), Missile(v_6)), Nation(x))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_7), Weapon(v_8)), Nation(x))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_9), Weapon(v_10)), Nation(x))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(x1), Missile(x2)), Hostile(Nation(v_11))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_12), Missile(v_13)), Hostile(Nation(x)))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_14), Weapon(v_15)), Hostile(Nation(x))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_16), Weapon(v_17)), Hostile(Nation(x))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_18), Missile(v_19)), Nono(x3))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_22), Weapon(v_23)), Nono(x3))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_26), Weapon(v_27)), Nono(x3))
To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3))
To(Sell(American(x1), Weapon(x2)), Hostile(Nation(x3))) ==> Be(American(x4), Criminal(x5))
Clauses KB
TRUE!
Table: real-time performances (in seconds) in the case of the command: turn off the alarm in the garage, subordinated
by the query: Colonel West is a criminal? (reasoning on the prior KB)
RESULTS: we had just −0.11% loss of performance on Raspberry Pi4 B (4GB) than respect to Intel i5-6600K
CONCLUSION: in a Caspar-based IoT system, the real-time performances will depend mostly on both responsiveness of
the actuators devices and quality of the Internet connection (by using Google API Cloud).
AD-CASPAR
Abductive-Deductive-Cognitive Architecture System Planned And Reactive
1. Due to KB expansions, large ones can be hard to manage.
2. CASPAR is just able of logic deduction, because it works only in
the presence of unifiable predicates. No feedback for possible close
results.
[1] C. Santoro, Carmelo Fabio Longo. “AD-CASPAR: Abductive-Deductive Cognitive Architecture based on Natural Language and First Order Logic Reasoning”. In: 4th
Workshop on Natural Language for Artificial Intelligence (NL4AI2020) co-located with the 19th International Conference of the Italian association for Artificial
Intelligence (AI*IA 2020). 2020.
• Affermative/Interrogative questions
• Margot said the truth about her life? → Logical form
• Auxiliary+Affermative Affermative/Interrogative questions
• Has Margot said the truth about her life? → Auxiliary removal → Logical form
✓ First dependency analysis: aux(said, Has)
Logical forms
• Plain
❖ Say_VBD(Margot_NNP(x1), About_IN(Truth_NN(x2), Her_PRP__Life_NN(x3)))
• Disambiguated
❖ State.v.01_VBD(Margot_NNP(x1), About_IN(Truth.n.03_NN(x2), Her_PRP__Animation.n.01_NN(x3)))
1. Dependencies analysys of the question
2. Splitting into chunks as it follow, starting from the delimiters [AUX] and [ROOT] which
are in general are always present in a wh-question:
[PRE AUX][AUX][POST AUX][ROOT][POST ROOT][COMPL ROOT]
3. Ricombination of the chunk by the means of production rules (QA-Shifter) respecting
the grammatical features of the current language, but in assertion-shape.
4. Introduction of additional adverbs (in case of where and when)+Dummy word
5. Simetric version of 3. for copular[1] verbs
Steps to deal with wh-questions (when, where, what, who, how):
[1] a non-transitive verb but identifying the subject with the object in the scope of a verbal phrases (like be)
Who could be the president of America?
The chunks are:
[PRE AUX][could][POST AUX][be][the president of America][COMPL ROOT]
with AUX= could, ROOT=be, POST_ROOT=the president of America
Such a wh-question will trigger a QA-Shifter rule to product the following two assertions:
Dummy could be the president of America
The president of America could be Dummy
The Translation Service, during parsing, will impose the Part-of-Speech DM to Dummy, whose
parsing is not expected by the Clauses Builder, thus it will discarded. At the end of this process, as
FOL expression of the query we'll have the following literal:
Be_VBZ(Biden_NNP(x1), x2)
Hight Clauses KB:
Be_VBZ(Biden_NNP(x3), Of_IN(President_NN(x4), America_NNP(x5))), ……
Reasoning (Backward-Chaining):
x1: x3, x2: Of_IN(President_NN(x4), America_NNP(x5))
which contains, in correspondence of the variable x2, the logic representation of the snippet:
president of America as possible and correct answer.
Low Clauses KB record (stored in a NoSQL db) for a sentence S in natural language:
▪ S: sentence
▪ C: nested definite clause of S
▪ Feat𝒄: vector labels of C
𝐶𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒𝐶 =
| ‫ځ‬(𝐅𝐞𝐚𝐭𝒒,𝐅𝐞𝐚𝐭𝒄)|
|𝐅𝐞𝐚𝐭𝒒|
Low Clauses KB High Clauses KB
definite clause C
𝐶𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒𝐶 < threshold
Realtime cognitive performances (in seconds) considering:
Question: Colonel West is a Criminal? (on the prior KB)
Confidence threshold: 0.6
where:
• West25: colonel West KB (expanded) made of 25 clauses
• West104: West25 + 78 unrelated clauses
• West303: West25 + 278 unrelated clauses
RESULTS
1. 0.416 > 0.378: it is expected such bias to be increased for larger
knowledge bases. Hence, LKB+HKB works better than just HKB.
2. It permits at the same time abduction as pre-stage of deduction,
in order to give back closer results also in presence of non-
successful reasoning.
SW-CASPAR
Semantic Web Cognitive Architecture System Planned And Reactive
1. KB expansion depends on clauses assertions order.
2. Such architectures are suitable only in scenarios with the Closed-
World Assumption.
CASPAR: meta-reasoning in a conceptual space based on
natural language processing, whose content is made of facts
and axioms in first-order logic with the closed-world
assumption
SW-CASPAR: meta-reasoning in a conceptual space
based on natural language processing, whose content is made
of shared ontologies with the open-world assumption
(Semantic Web)
Transposition
[1] C. F. Longo, C. Santoro, D. F. Santamaria, M. N. Asmundo, and D. Cantone. “Towards Ontological Interoperability of Cognitive IoT Agents, based
on Natural Language Processing”, in Intelligenza Artificiale, IOS Press, 2022. [ACCEPTED]
Modelling an ontology reflecting a domain, by means of a description in natural language of the
domain itself, with the aim of human-fashioned reasoning.
BIASES (lacks, noisy, etc)
• Manual
• Cooperative
• Semi-automatic
• Automatic (probably not feasible [1])
[1] A. Browarnik, O. Maimon, Ontology learning from text, in: The First International Conference on Big Data, Small Data, Linked
Data and Open Data, 2015.
Biases of Ontology Learning for the aim of human-like fashioned reasoning:
• Nonexistent objects representation
• An information and its complement present in the KB
• Deverbal Nominalization
• “Robert walked down the street” VS “Robert had a walk down the street”
• Deadjectival Nominalization
• “the friends are happy” VS “happy friends”.
• Others[1]
[1] F. Moltmann, Natural language ontology, 2017. URL: https://oxfordre.com/linguistics/view/10.1093/acrefore/9780199384655.001.0001/acrefore-
9780199384655-e-330. doi:10.1093/acrefore/9780199384655.013.330
L.O.D.O. (Linguistic Oriented Davidsonian Ontology). Can be
considered a foundational ontology, i.e., a specific type of
ontology designed to model high-level and domain
independent categories about the real world.
A set of triples in OWL 2 made by the following classes, properties and instances:
• Verb
• hasId (instance of Id)
• hasSubj (instance of Entity)
• hasObj (instance of Entity)
• hasAdv (instance of Adverb)
• hasPrep (instance of Preposition)
• Id
• Entity
• hasAdj (instance of Adjective)
• hasPrep (instance of Preposition)
• Adverb
• Adjective
• Preposition
• hasObj (instance of Entity)
A group of axioms (or part of them) implicitly created by SW-CASPAR, with the aim of
increasing the chances of reasoning. In the presence of the following FOL expression:
Subject:POS(x1) ∧ Cop:POS(e1, x1, x2) ∧ Object:POS(x2)
The Ontology Builder will assert the following SWRL axiom:
Subject(?x) -> Object(?x)
where Cop is copular verb (such as Be, for instance), i.e., an intransitive verb but
identifying its subject with its object; hence, in this case, the class membership of the
verb’s object will be inherited by the subject.
Such rules are implicitly asserted together with the Assignment Rules, to let a copular verb’s
subject inherits both adjectives and prepositions properties of the verb’s object.
Formally, considering Subject(?x) -> Object(?x), the corresponding legacy rule will be the
following:
Subject(?x2), Object(?x1), hasAdj(?x1, ?x3), Adjective(?x3) -> hasAdj(?x2, ?x3)
Subject(?x2), Object(?x1), hasPrep(?x1, ?x3), Adjective(?x3) -> hasPrep(?x2, ?x3)
• In the presence of an instance of Adjective, such rule asserts a new deadjectivated
instance of the latter as new membership of the adjective related noun. Formally:
Entity(?x1), hasAdj(?x1, ?x2), Adjective(?x2) -> Entity(?x2)
• In the presence of an instance of Verb, such rules assert a new deverbalized
instance of the latter having the same entities as the former, by the means of a
semantic production rules system supported by specific dictionary (in progress of
development).
The production rule of the Ontology Builder for such rules assertion takes into account the
following pattern:
Subject(𝑥𝑏𝑜𝑑𝑦) ∧... ⟹ Subject(𝑥𝑠𝑢𝑏𝑗 ) ∧ Object(𝑥𝑜𝑏𝑗) ∧ Cop(𝑒𝑐𝑜𝑝, 𝑥𝑠𝑢𝑏𝑗 , 𝑥𝑜𝑏𝑗)
permitting the formal assertion of the following pattern:
Subject(?𝑥𝑜𝑏𝑗), ... -> Object(?𝑥𝑜𝑏𝑗)
Example: when a dog awake the man, the dog is hungry
Result: in the presence of the verbal phrase: the dog awake the man, an instance related to
dog will acquire implicitly the membership to the class hungry.
• Value Giver statement: such a statement contributes to give a value to a data property
hasValue related to a specified individual, which is parsed by the Ontology Builder by
matching the following pattern of beliefs:
GND(FLAT, X, Y), ADJ(FLAT, X, "Equal"), PREP(FLAT, X, "To", S), VALUE(FLAT, S, V)
The property hasValue might be involved in comparison operations in the composition
of a SWRL axiom.
• Comparison Conditional: are parsed from sentences similarly to the Value Giver
Statement, but they will take place within the body of Implicative Copular Rules.
Robinson Crusoe is a patient
Robinson Crusoe has diastolic blood pressure equal to 150
When a patient has diastolic blood pressure greater than 140, the patient is hypertensive
+INTENT("Rinazina", P) / eval_sem(P, "Hypertensive")) » [say("Nope. Patient is hypertensive")]
+INTENT("Rinazina", P) » [exec_cmd("Rinazina", P), say("execution successful")]
Meta-Reasoning (Active Belief)
Command: Give Rinazina to P (P= patient) SW-CASPAR: +INTENT("Rinazina", P)
Give Rinazina to Robinson Crusoe
+INTENT("Rinazina", P) / eval_sem(P, "Hypertensive")) » [say("Nope. Patient is hypertensive")]
+INTENT("Rinazina", P) » [exec_cmd("Rinazina", P), say("execution successful")]
TRUE
Overall Evaluation and Conclusions
Together with soundness and time performances, two criteria from the state-of-
the-art have been also used for evalution with good results:
• MGC[1] (Minimal Cognitive Grid)
• SMM [2] (Standard Model of Mind)
CONCLUSION: CASPAR is a powerful startpoint to build complex cognitive
architectures, with an hybrid sub-symbolic/symbolic approach, based on natural
language processing.
[1] A. Lieto. Cognitive Design for Artificial Minds. Routledge, 2021. Chap. 3.
[2] Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A Standard Model of the Mind: Toward a Common Computational Framework across Artificial
Intelligence, Cognitive Science, Neuroscience, and Robotics. AI Magazine, 38(4), 13-26. https://doi.org/10.1609/aimag.v38i4.2744
Caspar:
AD-Caspar:
SW-Caspar:
https://github.com/fabiuslongo/pycaspar
https://github.com/fabiuslongo/ad-caspar
https://github.com/fabiuslongo/sw-caspar
A Family of Reactive-Cognitive Architectures based on Natural
Language Processing, as Decision-Making Helpers for IoT, in the
Closed- or Open-World Assumption
Ph.D. Thesis Defense
Candidate: Carmelo Fabio Longo
Advisor: Corrado Santoro
February 17, 2022

More Related Content

Similar to C.F. Longo Ph.D. Thesis Defence

Class 31: Deanonymizing
Class 31: DeanonymizingClass 31: Deanonymizing
Class 31: Deanonymizing
David Evans
 
TENSOR DECOMPOSITION WITH PYTHON
TENSOR DECOMPOSITION WITH PYTHONTENSOR DECOMPOSITION WITH PYTHON
TENSOR DECOMPOSITION WITH PYTHON
André Panisson
 
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inferenceJarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
PalGov
 
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inferenceJarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
PalGov
 
On the smallest enclosing information disk
 On the smallest enclosing information disk On the smallest enclosing information disk
On the smallest enclosing information disk
Frank Nielsen
 
Latent semantic analysis
Latent semantic analysisLatent semantic analysis
Latent semantic analysis
Kiarash Kiani
 
Breaking the Softmax Bottleneck: a high-rank RNN Language Model
Breaking the Softmax Bottleneck: a high-rank RNN Language ModelBreaking the Softmax Bottleneck: a high-rank RNN Language Model
Breaking the Softmax Bottleneck: a high-rank RNN Language Model
Ssu-Rui Lee
 
Pointing the Unknown Words
Pointing the Unknown WordsPointing the Unknown Words
Pointing the Unknown Words
hytae
 
CS571: Distributional semantics
CS571: Distributional semanticsCS571: Distributional semantics
CS571: Distributional semantics
Jinho Choi
 
Natural Language Processing(SupStat Inc)
Natural Language Processing(SupStat Inc)Natural Language Processing(SupStat Inc)
Natural Language Processing(SupStat Inc)
Vivian S. Zhang
 
Generating Chebychev Chaotic Sequence
Generating Chebychev Chaotic SequenceGenerating Chebychev Chaotic Sequence
Generating Chebychev Chaotic Sequence
Cheng-An Yang
 
Alastair Butler - 2015 - Round trips with meaning stopovers
Alastair Butler - 2015 - Round trips with meaning stopoversAlastair Butler - 2015 - Round trips with meaning stopovers
Alastair Butler - 2015 - Round trips with meaning stopovers
Association for Computational Linguistics
 

Similar to C.F. Longo Ph.D. Thesis Defence (12)

Class 31: Deanonymizing
Class 31: DeanonymizingClass 31: Deanonymizing
Class 31: Deanonymizing
 
TENSOR DECOMPOSITION WITH PYTHON
TENSOR DECOMPOSITION WITH PYTHONTENSOR DECOMPOSITION WITH PYTHON
TENSOR DECOMPOSITION WITH PYTHON
 
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inferenceJarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
 
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inferenceJarrar.lecture notes.aai.2011s.ch9.fol.inference
Jarrar.lecture notes.aai.2011s.ch9.fol.inference
 
On the smallest enclosing information disk
 On the smallest enclosing information disk On the smallest enclosing information disk
On the smallest enclosing information disk
 
Latent semantic analysis
Latent semantic analysisLatent semantic analysis
Latent semantic analysis
 
Breaking the Softmax Bottleneck: a high-rank RNN Language Model
Breaking the Softmax Bottleneck: a high-rank RNN Language ModelBreaking the Softmax Bottleneck: a high-rank RNN Language Model
Breaking the Softmax Bottleneck: a high-rank RNN Language Model
 
Pointing the Unknown Words
Pointing the Unknown WordsPointing the Unknown Words
Pointing the Unknown Words
 
CS571: Distributional semantics
CS571: Distributional semanticsCS571: Distributional semantics
CS571: Distributional semantics
 
Natural Language Processing(SupStat Inc)
Natural Language Processing(SupStat Inc)Natural Language Processing(SupStat Inc)
Natural Language Processing(SupStat Inc)
 
Generating Chebychev Chaotic Sequence
Generating Chebychev Chaotic SequenceGenerating Chebychev Chaotic Sequence
Generating Chebychev Chaotic Sequence
 
Alastair Butler - 2015 - Round trips with meaning stopovers
Alastair Butler - 2015 - Round trips with meaning stopoversAlastair Butler - 2015 - Round trips with meaning stopovers
Alastair Butler - 2015 - Round trips with meaning stopovers
 

Recently uploaded

The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
kalichargn70th171
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
ervikas4
 
Optimizing Your E-commerce with WooCommerce.pptx
Optimizing Your E-commerce with WooCommerce.pptxOptimizing Your E-commerce with WooCommerce.pptx
Optimizing Your E-commerce with WooCommerce.pptx
WebConnect Pvt Ltd
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Vince Scalabrino
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Ortus Solutions, Corp
 
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
dhavalvaghelanectarb
 
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdfTheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
Ortus Solutions, Corp
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
Zycus
 
Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
KrishnaveniMohan1
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
chandangoswami40933
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)
alowpalsadig
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
Anand Bagmar
 
Hands-on with Apache Druid: Installation & Data Ingestion Steps
Hands-on with Apache Druid: Installation & Data Ingestion StepsHands-on with Apache Druid: Installation & Data Ingestion Steps
Hands-on with Apache Druid: Installation & Data Ingestion Steps
servicesNitor
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
widenerjobeyrl638
 
Refactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contextsRefactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contexts
Michał Kurzeja
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
Tier1 app
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
safelyiotech
 

Recently uploaded (20)

The Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdfThe Comprehensive Guide to Validating Audio-Visual Performances.pdf
The Comprehensive Guide to Validating Audio-Visual Performances.pdf
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
 
Optimizing Your E-commerce with WooCommerce.pptx
Optimizing Your E-commerce with WooCommerce.pptxOptimizing Your E-commerce with WooCommerce.pptx
Optimizing Your E-commerce with WooCommerce.pptx
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
 
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
Strengthening Web Development with CommandBox 6: Seamless Transition and Scal...
 
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
 
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdfTheFutureIsDynamic-BoxLang-CFCamp2024.pdf
TheFutureIsDynamic-BoxLang-CFCamp2024.pdf
 
How GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdfHow GenAI Can Improve Supplier Performance Management.pdf
How GenAI Can Improve Supplier Performance Management.pdf
 
Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.Penify - Let AI do the Documentation, you write the Code.
Penify - Let AI do the Documentation, you write the Code.
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)Photoshop Tutorial for Beginners (2024 Edition)
Photoshop Tutorial for Beginners (2024 Edition)
 
Streamlining End-to-End Testing Automation
Streamlining End-to-End Testing AutomationStreamlining End-to-End Testing Automation
Streamlining End-to-End Testing Automation
 
Hands-on with Apache Druid: Installation & Data Ingestion Steps
Hands-on with Apache Druid: Installation & Data Ingestion StepsHands-on with Apache Druid: Installation & Data Ingestion Steps
Hands-on with Apache Druid: Installation & Data Ingestion Steps
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
 
Refactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contextsRefactoring legacy systems using events commands and bubble contexts
Refactoring legacy systems using events commands and bubble contexts
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSISDECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
DECODING JAVA THREAD DUMPS: MASTER THE ART OF ANALYSIS
 
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
Safelyio Toolbox Talk Softwate & App (How To Digitize Safety Meetings)
 

C.F. Longo Ph.D. Thesis Defence

  • 1. A Family of Reactive-Cognitive Architectures based on Natural Language Processing, as Decision-Making Helpers for IoT, in the Closed- or Open-World Assumption Ph.D. Thesis Defense Candidate: Carmelo Fabio Longo Advisor: Corrado Santoro February 17, 2022
  • 2.
  • 4. OS-1 "Samantha" from "Her" HAL 9000 from "2001: Space Odissey"
  • 5. • High deductive capabilities • Physical interaction with the environment • Meta-Reasoning in conceptual spaces for the task of decision- making • Session Awareness • History Awareness • Consciousness
  • 6.
  • 7. • High deductive capabilities • Physical interaction with the environment • Meta-Reasoning in conceptual spaces for the task of decision- making • Session Awareness • History Awareness • Consciousness
  • 8. [1] P. Thagard. “Critical thinking and informal logic: Neuropsychological perspectives.” In: Informal logic, 51 (2011), pp. 152–170. doi: https://doi.org/10.22329/il.v31i3.3398.
  • 9.
  • 10. GAP Fast environments with difficult-to-predict dynamics of the world High-Level Cognition Behaviours (Language, Complex Planning, Deductive Reasoning, etc.)
  • 11. • Language • Complex Planning • Deductive Reasoning • Differential equations • Statistics • Signal Processing Meta-Reasoning Reasoning in Conceptual Spaces
  • 13.
  • 14. [1] Carmelo Fabio Longo, Francesco Longo, Corrado Santoro “CASPAR: towards Decision Making Helpers Agents for IoT, based on Natural Language and First Order Logic Reasoning”, in Engineering Application of Artificial Intelligence, Elsevier, 2021.
  • 15. • Language • Complex Planning • Deductive Reasoning • Differential equations • Statistics • Signal Processing Meta-Reasoning CASPAR: Cognitive Architecture System Planned and Reactive
  • 16. • High deductive capabilities • Physical interaction with the environment • Meta-Reasoning in conceptual spaces for the task of decision-making • Session Awareness • History Awareness • Consciousness
  • 17.
  • 18. A pipeline of five modules with the task of taking a sound stream in natural language and translating it in a neo-davidsonian First Order Logic (FOL) expression inheriting the shape from the event-based formal representation of Davidson[1]. [1] D. Davidson, “The logical form of action sentences,” in The logic of decision and action, p. 81–95, University of Pittsburg Press, 1967. Brutus stabbed suddenly Caesar in the agora ∃e stabbed(e, Brutus, Caesar) ⋀ suddenly(e) ⋀ in(e, agora) e = davidsonian variable
  • 19. The MST Builder, has the purpose to build a novel semantic structure defined as Macro Semantic Table (MST). It summarizes in a canonical shape all the semantic features in a sentence, by leveraging a step-by- step dependencies analysis, in order to derive FOL expressions. MST(u) = {ACTIONS, VARLIST, PREPS, BINDS, COMPS, CONDS} ACTIONS = [(label𝑘, e𝑘, x𝑖 , x𝑗),...] VARLIST = [(x1, label1),...(x𝑛, label𝑛)] PREPS = [(label𝑗, (e𝑘 | x𝑖), x𝑗),...] BINDS = [(label𝑖, label𝑗),...] COMPS = [(label𝑖, label𝑗),...] CONDS = [e1, e2,...] where u = utterance in natural language
  • 20. ACTIONS = [(shines:VBZ, e1, x1 , x2), (is:VBZ, e2, x3 , x4)] VARLIST = [(x1,sun:NN), (x2, ?), (𝑒1, suddenly:RB), (x3, Robert:NN), (x4, happy:JJ), (e1, strongly:RB)] PREPS = [] BINDS = [] COMPS = [] CONDS = [e1] When the sun shines strongly, Robert is happy shines:VBZ(e1, x1, __) ⋀ sun:NN(x1)⋀ strongly:RB(e1) ⟹ is:VBZ(e2, x3 , x4) ⋀ Robert:NN(x3) ⋀ happy:JJ (x4) MST = FOL Builder
  • 21. • Managed by the BDI Framework PHIDIAS • The Speech-To-Text (STT) Front-End: Beliefs generator • Each Sensor Instance: it asserts specific beliefs related to sensors, such as: • Microphones • Temperature sensors • Video Capturing • Etc.
  • 22. Set the cooler at 27 degrees in the bedroom set:VB(d1, __, x1) ⋀ cooler:NN(x1) ⋀ at:IN(d1, x2) ⋀ 27:CD(x2) ⋀ degree:NNS(x2) ⋀ in:IN(x2, x3) ⋀ bedroom:NN(x3) INTENT(set, cooler, bedroom, (at 26 degree)) Translation Service Direct Command Parser
  • 23. Turn off the lights in the living room, when the temperature is 25 and the time is 12 be:VBZ(d2, x3, x4) ⋀ be:VBZ(d3, x5, x6) ⋀ temperature:NN(x3) ⋀ 25:CD(x4) ⋀ time:NN(x5) ⋀ 12:CD(x6) ⟹ turn:VB(d1, __, x1) ⋀ off:RP(d1) ⋀ light:NNS(x1) ⋀ in:IN(d1, x2) ⋀ living:NN(x2) ⋀ room:NN(x2) COND(337538, be, temperature, 25), COND(337538, be, time, 12) ROUTINE(337538, turn, light, living room, off) Translation Service Routines Command Parser
  • 24. COND(337538, be, temperature, 25), COND(337538, be, time, 12) ROUTINE(337538, turn, light, living room, off) Asserted by Sensor Instances: +SENSOR(be, temperature, 25) +SENSOR(be, time, 12) INTENT(turn, light, living room, off)
  • 25. shines:VBZ(e1, x1, __) ⋀ sun:NN(x1) ⋀ strongly:RB(e1) ⟹ is:VBZ(e2, x3 , x4) ⋀ Robert:NN(x3) ⋀ happy:JJ(x4) Definite clauses Builder shine:VBZ(sun:NN(x1), __) ⟹ be:VBZ(Robert:NNP(x3), happy:JJ(x4)) When the sun shines strongly, Robert is happy Translation Service
  • 26. • He likes to eat a bass • He likes to play the bass PROBLEM: where “bass” is intended as fish and where as musical instrument?
  • 27. INSIGHT: in a common sense words embedding, the bag of words (context) used for the gloss/examples related to a synset comprising a lemma, should be likely vectorial closer (among all synsets comprising that lemma) to another bag of words making effective usage of such a lemma (in a similar context).
  • 28. Unsupervised [1] naive “bag of words” strategy, taking in account of the higher word2vect similarity (sim) for each lemma l of the sentence S (depending of the richness of the domain): [1] Navigli, R., 2009. Word sense disambiguation: A survey. ACM Comput. Surv. 41 (2), 10. http://dx.doi.org/10.1145/1459352.1459355.
  • 29. • He likes to eat a bass Like.v.05:VBZ_Feed.v.06:VB(He:PRP(x1), Sea_bass.n.01:NN(x2)) • He likes to play the bass Like.v.05:VBZ_Play.v.18:VB(He:PRP(x1), Bass.n.07:NN(x2)) Where the glosses are: Sea_bass.n.01: the lean flesh of a saltwater fish of the family Serranidae Bass.n.07: the member with the lowest range of a family of musical instruments
  • 30.
  • 31. # turn off the light in the kitchen +INTENT(X, "light", "kitchen", T) / lemma_in_syn(X, "change_state.v.01") >> [exec_cmd("change_state.v.01", "light", "kitchen", T) # turn off the light in the garage +INTENT(X, "alarm", "garage", T) / (lemma_in_syn(X, "change_state.v.01") & eval_cls("At_IN(Be_VBP(Person_NN(x1), __), Home_NN(x2))")) >> [exec("change_state.v.01", "alarm", "garage", T)] where: • X = Verb, T = parameters • eval_cls(C) is an Active belief = True if C can be deducted from Clauses KB, False otherwise • At_IN(Be_VBP(Person_NN(x1), __), Home_NN(x2)) represents the sentence: A person is at home
  • 32.
  • 33. • Nono is an hostile nation • Colonel West is American • missiles are weapons • Colonel West sells missiles to Nono • When an American sells weapons to a hostile nation, that American is a criminal • Be(Nono(x1), Hostile(Nation(x2))) • Be(Colonel_West(x1), American(x2)) • Be(Missile(x1), Weapon(x2)) • To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) • To(Sell(American(x1), Weapon(x2)), Hostile(Nation(x3)) ⟹ Be(American(x4), Criminal(x5)) Clauses KB Question: Colonel West is a criminal? Query: Be(Colonel_West(x1), Criminal(x2)) ? FALSE! [1] P.N. Stuart J. Russell, Artificial Intelligence: A Modern Approach, Pearson, 2010, Chapter 9.3.
  • 34. Roy_NNP(y) ⟹ man_NN(y) Assignment Rule Roy_NNP(y) ⟹ man_NN(y) talk_VBZ(Roy_NNP(x1), woman_NN(x2)) Clauses KB: talk_VBZ(Roy_NNP(x1), woman_NN(x2)) ⟹ talk_VBZ(man_NN(x1), woman_NN(x2)) Assert implicitly be:VBZ(e1, x1, x2) ∧ Roy:NNP(x1) ∧ man:NN(x2) Roy is a man If the lemma be is disambiguated with be.v.01, which is defined by its gloss as: have the quality of being [something], we can assert implicitly the following clause: (Roy talk with a woman) (Roy talk with a woman ⟹ A man talk with a woman)
  • 35. Having a simple KB as it follows: 𝑃1(𝐺1(x1)) ⋀ 𝑃2(𝐺2(x2)) ⟹ 𝑃3(𝐹3(x3)) 𝑃1(𝐹1(x1)) 𝑃2(𝐹2(x2)) Clause Conceptual Generalization of KB is: 𝑃1(𝐺1 1(x1)) ⋀ 𝑃2(𝐺2(x2)) ⟹ 𝐹3(x3) 𝐹1(x1) 𝐹2(x2) 𝑃1, 𝑃2, 𝑃3 can be considered as modifiers of respectively 𝐹1, 𝐹2, 𝐹3. The body (or left hand-side) of the implication is unchanged to hold the quality of the axiom.
  • 36. When the sun shines hard, Barbara drinks slowly a fresh lemonade Hard(Shine(Sun(x1), __)) ⟹ Drink(Barbara(x3), Lemonade(x4)) Hard(Shine(Sun(x1), __)) ⟹ Slowly(Drink(Barbara(x3), Lemonade(x4))) Hard(Shine(Sun(x1), __)) ⟹ Drink(Barbara(x3), Fresh(Lemonade(x4))) Hard(Shine(Sun(x1), __)) ⟹ Slowly(Drink(Barbara(x3), Fresh(Lemonade(x4)))) Generalizations
  • 37.
  • 38.
  • 39.
  • 40. Be(Nono(x1), Nation(x2)) Be(Nono(x1), Hostile(Nation(x2))) Nono(x) ==> Nation(x) Nono(x) ==> Hostile(Nation(x)) Be(Colonel_West(x1), American(x2)) Colonel_West(x) ==> American(x) New Query: Be(American(x4), Criminal(x5)) ? (NESTED REASONING) Be(Missile(x1), Weapon(x2)) Missile(x) ==> Weapon(x) Sell(Colonel_West(x1), Missile(x2)) ==> Sell(American(v_0), Missile(x2)) Sell(Colonel_West(x1), Missile(x2)) ==> Sell(American(x), Weapon(v_1)) Sell(Colonel_West(x1), Missile(x2)) ==> Sell(Colonel_West(x1), Weapon(v_2)) Sell(Colonel_West(x1), Missile(x2)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(x1), Missile(x2)), Nation(v_4)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_5), Missile(v_6)), Nation(x)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_7), Weapon(v_8)), Nation(x)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_9), Weapon(v_10)), Nation(x)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(x1), Missile(x2)), Hostile(Nation(v_11)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_12), Missile(v_13)), Hostile(Nation(x))) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_14), Weapon(v_15)), Hostile(Nation(x)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_16), Weapon(v_17)), Hostile(Nation(x)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_18), Missile(v_19)), Nono(x3)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(American(v_22), Weapon(v_23)), Nono(x3)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) ==> To(Sell(Colonel_West(v_26), Weapon(v_27)), Nono(x3)) To(Sell(Colonel_West(x1), Missile(x2)), Nono(x3)) To(Sell(American(x1), Weapon(x2)), Hostile(Nation(x3))) ==> Be(American(x4), Criminal(x5)) Clauses KB TRUE!
  • 41. Table: real-time performances (in seconds) in the case of the command: turn off the alarm in the garage, subordinated by the query: Colonel West is a criminal? (reasoning on the prior KB) RESULTS: we had just −0.11% loss of performance on Raspberry Pi4 B (4GB) than respect to Intel i5-6600K CONCLUSION: in a Caspar-based IoT system, the real-time performances will depend mostly on both responsiveness of the actuators devices and quality of the Internet connection (by using Google API Cloud).
  • 43. 1. Due to KB expansions, large ones can be hard to manage. 2. CASPAR is just able of logic deduction, because it works only in the presence of unifiable predicates. No feedback for possible close results.
  • 44. [1] C. Santoro, Carmelo Fabio Longo. “AD-CASPAR: Abductive-Deductive Cognitive Architecture based on Natural Language and First Order Logic Reasoning”. In: 4th Workshop on Natural Language for Artificial Intelligence (NL4AI2020) co-located with the 19th International Conference of the Italian association for Artificial Intelligence (AI*IA 2020). 2020.
  • 45. • Affermative/Interrogative questions • Margot said the truth about her life? → Logical form • Auxiliary+Affermative Affermative/Interrogative questions • Has Margot said the truth about her life? → Auxiliary removal → Logical form ✓ First dependency analysis: aux(said, Has) Logical forms • Plain ❖ Say_VBD(Margot_NNP(x1), About_IN(Truth_NN(x2), Her_PRP__Life_NN(x3))) • Disambiguated ❖ State.v.01_VBD(Margot_NNP(x1), About_IN(Truth.n.03_NN(x2), Her_PRP__Animation.n.01_NN(x3)))
  • 46. 1. Dependencies analysys of the question 2. Splitting into chunks as it follow, starting from the delimiters [AUX] and [ROOT] which are in general are always present in a wh-question: [PRE AUX][AUX][POST AUX][ROOT][POST ROOT][COMPL ROOT] 3. Ricombination of the chunk by the means of production rules (QA-Shifter) respecting the grammatical features of the current language, but in assertion-shape. 4. Introduction of additional adverbs (in case of where and when)+Dummy word 5. Simetric version of 3. for copular[1] verbs Steps to deal with wh-questions (when, where, what, who, how): [1] a non-transitive verb but identifying the subject with the object in the scope of a verbal phrases (like be)
  • 47. Who could be the president of America? The chunks are: [PRE AUX][could][POST AUX][be][the president of America][COMPL ROOT] with AUX= could, ROOT=be, POST_ROOT=the president of America Such a wh-question will trigger a QA-Shifter rule to product the following two assertions: Dummy could be the president of America The president of America could be Dummy
  • 48. The Translation Service, during parsing, will impose the Part-of-Speech DM to Dummy, whose parsing is not expected by the Clauses Builder, thus it will discarded. At the end of this process, as FOL expression of the query we'll have the following literal: Be_VBZ(Biden_NNP(x1), x2) Hight Clauses KB: Be_VBZ(Biden_NNP(x3), Of_IN(President_NN(x4), America_NNP(x5))), …… Reasoning (Backward-Chaining): x1: x3, x2: Of_IN(President_NN(x4), America_NNP(x5)) which contains, in correspondence of the variable x2, the logic representation of the snippet: president of America as possible and correct answer.
  • 49. Low Clauses KB record (stored in a NoSQL db) for a sentence S in natural language: ▪ S: sentence ▪ C: nested definite clause of S ▪ Feat𝒄: vector labels of C 𝐶𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒𝐶 = | ‫ځ‬(𝐅𝐞𝐚𝐭𝒒,𝐅𝐞𝐚𝐭𝒄)| |𝐅𝐞𝐚𝐭𝒒| Low Clauses KB High Clauses KB definite clause C 𝐶𝑜𝑛𝑓𝑖𝑑𝑒𝑛𝑐𝑒𝐶 < threshold
  • 50.
  • 51. Realtime cognitive performances (in seconds) considering: Question: Colonel West is a Criminal? (on the prior KB) Confidence threshold: 0.6 where: • West25: colonel West KB (expanded) made of 25 clauses • West104: West25 + 78 unrelated clauses • West303: West25 + 278 unrelated clauses RESULTS 1. 0.416 > 0.378: it is expected such bias to be increased for larger knowledge bases. Hence, LKB+HKB works better than just HKB. 2. It permits at the same time abduction as pre-stage of deduction, in order to give back closer results also in presence of non- successful reasoning.
  • 52. SW-CASPAR Semantic Web Cognitive Architecture System Planned And Reactive
  • 53. 1. KB expansion depends on clauses assertions order. 2. Such architectures are suitable only in scenarios with the Closed- World Assumption.
  • 54. CASPAR: meta-reasoning in a conceptual space based on natural language processing, whose content is made of facts and axioms in first-order logic with the closed-world assumption SW-CASPAR: meta-reasoning in a conceptual space based on natural language processing, whose content is made of shared ontologies with the open-world assumption (Semantic Web) Transposition [1] C. F. Longo, C. Santoro, D. F. Santamaria, M. N. Asmundo, and D. Cantone. “Towards Ontological Interoperability of Cognitive IoT Agents, based on Natural Language Processing”, in Intelligenza Artificiale, IOS Press, 2022. [ACCEPTED]
  • 55.
  • 56. Modelling an ontology reflecting a domain, by means of a description in natural language of the domain itself, with the aim of human-fashioned reasoning. BIASES (lacks, noisy, etc) • Manual • Cooperative • Semi-automatic • Automatic (probably not feasible [1]) [1] A. Browarnik, O. Maimon, Ontology learning from text, in: The First International Conference on Big Data, Small Data, Linked Data and Open Data, 2015.
  • 57. Biases of Ontology Learning for the aim of human-like fashioned reasoning: • Nonexistent objects representation • An information and its complement present in the KB • Deverbal Nominalization • “Robert walked down the street” VS “Robert had a walk down the street” • Deadjectival Nominalization • “the friends are happy” VS “happy friends”. • Others[1] [1] F. Moltmann, Natural language ontology, 2017. URL: https://oxfordre.com/linguistics/view/10.1093/acrefore/9780199384655.001.0001/acrefore- 9780199384655-e-330. doi:10.1093/acrefore/9780199384655.013.330
  • 58. L.O.D.O. (Linguistic Oriented Davidsonian Ontology). Can be considered a foundational ontology, i.e., a specific type of ontology designed to model high-level and domain independent categories about the real world.
  • 59. A set of triples in OWL 2 made by the following classes, properties and instances: • Verb • hasId (instance of Id) • hasSubj (instance of Entity) • hasObj (instance of Entity) • hasAdv (instance of Adverb) • hasPrep (instance of Preposition) • Id • Entity • hasAdj (instance of Adjective) • hasPrep (instance of Preposition) • Adverb • Adjective • Preposition • hasObj (instance of Entity)
  • 60. A group of axioms (or part of them) implicitly created by SW-CASPAR, with the aim of increasing the chances of reasoning. In the presence of the following FOL expression: Subject:POS(x1) ∧ Cop:POS(e1, x1, x2) ∧ Object:POS(x2) The Ontology Builder will assert the following SWRL axiom: Subject(?x) -> Object(?x) where Cop is copular verb (such as Be, for instance), i.e., an intransitive verb but identifying its subject with its object; hence, in this case, the class membership of the verb’s object will be inherited by the subject.
  • 61. Such rules are implicitly asserted together with the Assignment Rules, to let a copular verb’s subject inherits both adjectives and prepositions properties of the verb’s object. Formally, considering Subject(?x) -> Object(?x), the corresponding legacy rule will be the following: Subject(?x2), Object(?x1), hasAdj(?x1, ?x3), Adjective(?x3) -> hasAdj(?x2, ?x3) Subject(?x2), Object(?x1), hasPrep(?x1, ?x3), Adjective(?x3) -> hasPrep(?x2, ?x3)
  • 62. • In the presence of an instance of Adjective, such rule asserts a new deadjectivated instance of the latter as new membership of the adjective related noun. Formally: Entity(?x1), hasAdj(?x1, ?x2), Adjective(?x2) -> Entity(?x2) • In the presence of an instance of Verb, such rules assert a new deverbalized instance of the latter having the same entities as the former, by the means of a semantic production rules system supported by specific dictionary (in progress of development).
  • 63. The production rule of the Ontology Builder for such rules assertion takes into account the following pattern: Subject(𝑥𝑏𝑜𝑑𝑦) ∧... ⟹ Subject(𝑥𝑠𝑢𝑏𝑗 ) ∧ Object(𝑥𝑜𝑏𝑗) ∧ Cop(𝑒𝑐𝑜𝑝, 𝑥𝑠𝑢𝑏𝑗 , 𝑥𝑜𝑏𝑗) permitting the formal assertion of the following pattern: Subject(?𝑥𝑜𝑏𝑗), ... -> Object(?𝑥𝑜𝑏𝑗) Example: when a dog awake the man, the dog is hungry Result: in the presence of the verbal phrase: the dog awake the man, an instance related to dog will acquire implicitly the membership to the class hungry.
  • 64. • Value Giver statement: such a statement contributes to give a value to a data property hasValue related to a specified individual, which is parsed by the Ontology Builder by matching the following pattern of beliefs: GND(FLAT, X, Y), ADJ(FLAT, X, "Equal"), PREP(FLAT, X, "To", S), VALUE(FLAT, S, V) The property hasValue might be involved in comparison operations in the composition of a SWRL axiom. • Comparison Conditional: are parsed from sentences similarly to the Value Giver Statement, but they will take place within the body of Implicative Copular Rules.
  • 65. Robinson Crusoe is a patient Robinson Crusoe has diastolic blood pressure equal to 150 When a patient has diastolic blood pressure greater than 140, the patient is hypertensive
  • 66.
  • 67.
  • 68. +INTENT("Rinazina", P) / eval_sem(P, "Hypertensive")) » [say("Nope. Patient is hypertensive")] +INTENT("Rinazina", P) » [exec_cmd("Rinazina", P), say("execution successful")] Meta-Reasoning (Active Belief) Command: Give Rinazina to P (P= patient) SW-CASPAR: +INTENT("Rinazina", P)
  • 69. Give Rinazina to Robinson Crusoe
  • 70. +INTENT("Rinazina", P) / eval_sem(P, "Hypertensive")) » [say("Nope. Patient is hypertensive")] +INTENT("Rinazina", P) » [exec_cmd("Rinazina", P), say("execution successful")] TRUE
  • 71. Overall Evaluation and Conclusions
  • 72. Together with soundness and time performances, two criteria from the state-of- the-art have been also used for evalution with good results: • MGC[1] (Minimal Cognitive Grid) • SMM [2] (Standard Model of Mind) CONCLUSION: CASPAR is a powerful startpoint to build complex cognitive architectures, with an hybrid sub-symbolic/symbolic approach, based on natural language processing. [1] A. Lieto. Cognitive Design for Artificial Minds. Routledge, 2021. Chap. 3. [2] Laird, J. E., Lebiere, C., & Rosenbloom, P. S. (2017). A Standard Model of the Mind: Toward a Common Computational Framework across Artificial Intelligence, Cognitive Science, Neuroscience, and Robotics. AI Magazine, 38(4), 13-26. https://doi.org/10.1609/aimag.v38i4.2744
  • 74. A Family of Reactive-Cognitive Architectures based on Natural Language Processing, as Decision-Making Helpers for IoT, in the Closed- or Open-World Assumption Ph.D. Thesis Defense Candidate: Carmelo Fabio Longo Advisor: Corrado Santoro February 17, 2022