Milano Digital Week 2019, Lecture 16th of March. Matteo Palmonari and Federico Bianchi

1. Artificial thinking:
can machines reason
with analogies?
Pensiero artificiale:
le macchine sanno
“ragionare” per
analogia?
Matteo Palmonari and Federico Bianchi
University of Milan-Bicocca
INSID&S Lab
Interaction and Semantics for
Innovation with Data & Services

2. Impact of Modern Machine Learning on
Problem Solving / Games
2

3. A Multi-faceted Intelligence
3
…

4. Research Trends: Towards Human-level AIc
Humans learn faster and without limited training “data”
○ Transfer learning, few-shot learning, combination of symbolic knowledge and learning
Different humans’ cognitive skills are highly connected and not learnt
independently
○ Transfer learning, multi-modal learning (e.g., text + images, verbal + non-verbal communication,
images + emotions, …); multi-task learning (e.g., ~ one model for different tasks); combination
of analogical and logical reasoning; cognitively grounded architectures
Humans can imagine and reason about things and events that do not exist, and
at higher level of depth and abstraction
○ Imagination machines, generative models, what-if question answering, counterfactuals
4

5. Impact of Modern Machine Learning on
Problem Solving / Games
5

6. Research Trends: Towards Human-level AIc
Humans learn faster and without limited training “data”
○ Transfer learning, few-shot learning, combination of symbolic knowledge and learning
Different humans’ cognitive skills are highly connected and not learnt
independently
○ Transfer learning, multi-modal learning (e.g., text + images, verbal + non-verbal communication,
images + emotions, …); multi-task learning (e.g., ~ one model for different tasks); combination
of analogical and logical reasoning; cognitively grounded architectures
Humans can imagine and reason about things and events that do not exist,
and at higher level of depth and abstraction
○ Imagination machines, generative models, what-if question answering, counterfactuals
6

7. Imagination and AI in Creative Processes
7

8. Research Trends: Towards Human-level AI
Humans learn faster and without limited training “data”
○ Transfer learning, few-shot learning, combination of symbolic knowledge and learning
Different humans’ cognitive skills are highly connected and not learnt
independently
○ Transfer learning, multi-modal learning (e.g., text + images, verbal + non-verbal communication,
images + emotions, …); multi-task learning (e.g., ~ one model for different tasks); combination
of analogical and logical reasoning; cognitively grounded architectures
Humans can imagine and reason about things and events that do not exist, and
at higher level of depth and abstraction
○ Imagination machines, generative models, what-if question answering, counterfactuals
8

9. Research Trends: Towards Human-level AI
Humans learn faster and without limited training “data”
○ Transfer learning, few-shot learning, combination of symbolic knowledge and learning
Different humans’ cognitive skills are highly connected and not learnt
independently
○ Transfer learning, multi-modal learning (e.g., text + images, verbal + non-verbal communication,
images + emotions, …); multi-task learning (e.g., ~ one model for different tasks); combination
of analogical and logical reasoning; cognitively grounded architectures
Humans can imagine and reason about things and events that do not exist, and
at higher level of depth and abstraction
○ Imagination machines, generative models, what-if question answering, counterfactuals
9

10. Combining Knowledge Representation and Learning
10

11. Traditional and Distributional Semantics

12. Traditional Semantics: Interpretation and Inference
Intuitive interpretation (remark: it is a bit more complex than this)
● Barack Obama: a name that represents an object
● Politician: a concept that represents a set of objects
● Married to: a relation that represent a connection between these
objects
Interpretation of sentences and inference
● Barack Obama is married to Michelle Obama is true if the the
objects represented by Barack Obama and Michelle Obama
belong to the set of married couples
● All people that are friends of the husband are also friends of the
wife. Jay-Z is friend of Barack Obama →→→ Jay-Z is friend of
Michelle Obama

13. Traditional Semantics: Interpretation and Inference
Intuitive interpretation (remark: it is a bit more complex than this)
● Barack Obama: a name that represents an object
● Politician: a concept that represents a set of objects
● Married to: a relation that represent a connection between these
objects
Interpretation of sentences and inference
● Barack Obama is married to Michelle Obama is true if the the
objects represented by Barack Obama and Michelle Obama
belong to the set of married couples
● All people that are friends of the husband are also friends of the
wife. Jay-Z is friend of Barack Obama →→→ Jay-Z is friend of
Michelle Obama
Difficult to answer other questions:
● Who’s the US president most similar to Barack Obama?
● Which concept is similar to Politician?
● Who’s the equivalent of Barack Obama in France?

14. From Words to Meaning: Natural Language
The first approaches that were developed to study natural language required deep
knowledge about linguistic. An example:
https://towardsdatascience.com/word2vec-a-baby-step-in-deep-learning-but-a-giant-leap-towards-natural-language-processing-40fe4e8602ba
Difficult to capture meaning, strong linguistic skills are needed (e.g., many prefixes
and suffixes)

15. From Words To Meaning: Word Hierarchies
Wordnet [Miller, 1995]
organize words in a hierarchy.
Motorcar and Go-kart are
similar because they share
the fact of being motor
vehicle
Strict definition of similarity

16. From Words To Meaning: Meaning in Context
(From Lenci & Evert): what’s the meaning of ‘bardiwac’?
‘Bardiwac’ is a heavy red alcoholic beverage made from grape
● He handed her glass of bardiwac.
● Beef dishes are made to complement the bardiwacs.
● Nigel staggered to his feet, face flushed from too much bardiwac.
● Malbec, one of the lesser-known bardiwac grapes, responds well to
Australia’s sunshine.
● I dined on bread and cheese and this excellent bardiwac.
● The drinks were delicious: blood-red bardiwac as well as light, sweet
Rhenish.

17. From Words To Meaning: Meaning in Context
● “The meaning of a word is its use in the language” (Wittgenstein, 1953)
● “You shall know a word by the company it keeps” (Firth, 1957)
Distributional Hypothesis: similar words tend to appear in
similar contexts:
‘Bardiwac’ appears in drinking-related contexts

18. From Words To Meaning: Vector Space
The Vector Space is a mathematical object that can be used to model and
represent words. One word is represented by one vector in the vector space
cat
dog
house
Intuition
put similar words close to each other.
The closer two words are in the vector
space the more similar they are

19. From Words To Meaning: Word2Vec
Word2Vec [Mikolov+, 2013] is an algorithms that can be used to generate vector
representations of words from text: word embeddings.
cat
dog
similar words corresponds
to similar vectors
The big black cat eats its food.
My little black cat sleeps all day.
Sometimes my dog eats too much!
Word2Vec considers the context of appearance of each word in text and generates
vector representations from that.

20. Word Embeddings: Examples in 2D
http://ai-distillery.io

21. Word Embeddings: Similarity/Distance
http://ai-distillery.io

22. Analogies
Analogy: similarity/comparison between one thing and another
Propositional Analogy:
Soccer Player : Soccer Field = Driver : Autodrome

23. Word Embeddings: Analogical Reasoning
France
Paris
Rome
Italy
Word
representation
maintain
linguistic
regularities that
are useful to do
analogical
reasoning

24. Word Embeddings: Analogical Reasoning
v(king) - v(male) + v(female) ≈ v(queen)
Reasoning = Operation in the
vector space

25. Words Change Meaning Over Time
Di Carlo, V., Bianchi, F. & Palmonari, M. (2019). Training Temporal Word
Embeddings with a Compass. In AAAI.

26. Temporal Word Analogies: Examples

27. • Word meanings are constantly evolving, reflecting the continuous
change of the world and the needs of its speakers
(Kulkarni et al. 2015)
For example:
•apple:
fruit → computer → smartphone
•trump:
real estate → television → POTUS
Language changes across time…

28. Temporal Word Embeddings
● Temporal word embeddings are vector representations of words during
specific temporal intervals (e.g. the year 2001, the day 3/28/2018)
● They are learned from diachronic text corpora, divided in multiple
temporal slices (e.g. news articles, social posts)
1999
clinton, 2001
clinton, 2000
clinton, 1999
2000
2001

29. Temporal Word Embeddings with a Compass
The vector spaces are already aligned after the training!
• TWEC is based on an Atemporal Compass
Di Carlo, V., Bianchi, F. & Palmonari, M. (2019).
Training Temporal Word Embeddings with a
Compass. AAAI

30. TWEC: Temporal Word Embeddings with a Compass
president
senator
hillary
foundation
administration
texas
george
bill
clinton,1999
bush,1999
bush,2001
clinton,2001

31. TWEC
Example
Each point is the
representation of
a president in a
given year (e.g.,
bush_2001)

32. From Words to Entities
Bianchi, F. and Palmonari, M. (2017, November). Joint Learning of Entity and Type
Embeddings for Analogical Reasoning with Entities. In Proceedings of the 1st
Workshop on Natural Language for AI, 16th International Conference of the Italian
Association for Artificial Intelligence (AI*IA).

33. Words are Ambiguous
v(“Rome”) - v(“Italy”) + v(“France”) ≈ v(“Paris”)
Which Paris?

34. Representations of The Objects of The World
Paris is the most beautiful city in Texas
<Pairs,_Texas> is the most beautiful city in <Texas>
Disambiguation algorithms
Vector Representations of
Entities
Unique symbol to
identify Paris in Texas

35. Words vs Entities
● Paris
● Texas
● Greece
● France
Pairs has multiple meanings (Paris in Texas, Paris in
mythology, Paris in France…)

36. Words vs Entities: Words

37. Words vs Entities: Entities

38. Analogies with Entities
Exploring Knowledge with Entities

39. Humans and Analogical Reasoning

40. Subjective vs Objective Analogical Reasoning
Pizza : Italy = X : France
Computational analogical reasoning Human analogical reasoning
● Answer is based on
personal experience
● Answer is based on
computation, for example
vector computation
X = [Croissant, Omelette, Baguette, Escargot, …]Generally one answer
Answer is given by numerical
factor
Which are the factors that can
influence the answer?

41. ARGO: Analogical Reasoning Game with a purpOse
We built ARGO, in collaboration with Cefriel, to collect
analogical answers
http://argo.disco.unimib.it
ARGO is a game that allows users to play
an analogical game with other people:
● 3 categories: VIPs, Music, Movies.
● Select an answer you think the other
player might give
● Get points if you and the other player
give the same answer

45. Experimental Evaluation: MeetMeTonight
● On the 29th of September 2018 ARGO was presented at the
MeetMeTonight Event:
○ 90 players
○ 1200 analogical answers

46. Type Sharing Factor

47. Relational Factor

48. Popularity Factor

49. Time Representations
Bianchi, F., Palmonari, M., & Nozza, D. (2018, October). Towards Encoding Time in
Text-Based Entity Embeddings. In International Semantic Web Conference (pp. 56-71).

50. Textual Descriptions of Time Periods via Events

51. Textual Descriptions of Time Periods via Events
“The succession of events is an inherent property of our time
perception. Memory is necessary, and the order of these
events is fundamental”
Snaider&al. 2012, Cognitive Systems Research

52. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description

53. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description

54. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
Adolf Hitler
Nazi Germany
World War II
4 3 6 2 3
5 1 2 9 2
1 2 8 4 1

55. Embedding Years from Event Descriptions
A year is represented by the set of entities taking part in the year’s events
The year vector is the average of the entities’ vectors found inside the description
Adolf Hitler 4 3 6 2 3
Nazi Germany 5 1 2 9 2
World War II 1 2 8 4 1
1941
9 2 3 5 5
AVG
Bianchi, F., Palmonari, M., & Nozza, D. (2018,
October). Towards Encoding Time in Text-Based
Entity Embeddings. In ISWC.

56. Embedded Representations vs. Natural Time Flow
191X
years
201X
years
PCA in 1D vs. natural order of years: Kendall τ = 0.80 and Spearman Rank correlation coefficient = 0.94
Good resemblance of natural time flow!
2D projection (PCA)
1D projection (PCA)

57. Conclusions
● Algorithms can model conceptual representations of
words and entities
● Similarity and Analogy are key aspects of reasoning
○ Softer and approximate reasoning over data