After the amazing breakthroughs of machine learning (deep learning or otherwise) in the past decade, the shortcomings of machine learning are also becoming increasingly clear: unexplainable results, data hunger and limited generalisability are all becoming bottlenecks. In this talk we will look at how the combination with symbolic AI (in the form of very large knowledge graphs) can give us a way forward, towards machine learning systems that can explain their results, that need less data, and that generalise better outside their training set. -- Frank van Harmelen leads the Knowledge Representation & Reasoning group in the CS Department of the VU University Amsterdam. He is also Principal investigator of the Hybrid Intelligence Centre, a 20Μ€, 10 year collaboration between researchers at 6 Dutch universities into AI that collaborates with people instead of replacing them. --

1. Systems that learn and reason
Frank van Harmelen
Knowledge Representation & Reasoning Group
Vrije Universiteit Amsterdam
Creative Commons License
CC BY 3.0:
Allowed to copy, redistribute
remix & transform
But must attribute
1

2. Direction of travel in modern AI
“making artificial intelligence more human-
centred”
AI systems should “support people and be
competent partners”
But:
“current AI systems are often incompetent
because
they lack background and contextual
knowledge” 2

3. So what’s holding AI back?
For a long time,
AI researchers have locked themselves
in one of two towers 3
“current AI systems are often incompetent because
they lack background and contextual knowledge”
and “they cannot explain their actions”

4. AI: a tale of
two towers?
Statistical AI Symbolic AI

5. AI: two religions?

6. But our brain uses both …
6

7. In our brain (roughly)
Data
Driven
Knowledge
Driven

8. Symbolic
Knowledge
Logic
Reasoning
Connectionist
Data
Statistics
Learning
The AI pendulum!
8

9. So let’s compare
9

10. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
10

11. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
Very large knowledge graphs:
Billions of facts and rules,
But expensive to build & maintain:
- Crowd sourcing
11

12. Strengths & Weaknesses
10M training samples
Symbolic Connectionist
Construction Human effort Data hunger
Scalable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
4.8M training games
12

13. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
worse with
more data
worse with
less data
13

14. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff

15. Strengths & Weaknesses
Symbolic Statistical
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
15
“black box problem”

16. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
quality

generality  16

17. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
17
Class: 793
Label: n04209133 (shower cap)
Certainty: 99.7%
Problem:
“out of sample generalizability”

18. Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Generalisable Performance cliff Performance cliff
18

19. Can we get them
to collaborate?
19

20. Learning to add facts to knowledge graph
(“knowledge graph completion”)
Rolling
Stones
Angi
Beat It
Jackson
Publish
20
Angi
Rolling
Stones
Publish
From: Predict:
Yesterday
Publish
Beatles
…..
Publish
…..

21. prediction
algorithm
21
Learning to add facts to a knowledge graph
(“knowledge graph completion”)

22. Example: Inductive Logic Programming,
rule mining
(anyBURL on knowledge graphs)
22
parent(Mary,Vicky).
parent(Mary,Andre).
parent(Carrey,Vickey).
mother(Mary,Vicky).
mother(Mary,Andy).
father(Carrey,Vickey).
father(Carrey,Andy).
parent(X,Y) :- mother(X,Y).
parent(X,Y) :- father(X,Y)
parent(Carrey,Andy).

23. Inductive Logic Programming:
application
23
Labels
Facts Theory
Active
Inactive

24. Informed ML using KR
queen
crown
wears
24
shower
cap
?
Predict Select
crown?
showercap?

25. Explainable ML using KR
queen
wears
25
shower
cap
?
Predict Explain
crown?

27. Context aware ML using KR
(informed ML)
See survey of 100+ systems in Von Rueden et al, Learning, 2019
flower?
cushion?
“Parts of a chair are:
cushion and armrest”
“Given the context of chair,
a cushion is much more likely
than a flower”
P(cushion|chair) >> P(flower|chair)

28. Learning intermediate abstractions
:- see( , 3), see( , 5), add(3,5,8).
End-to-end:
2x 784
inputs
19 outputs
8

29. Learning intermediate abstractions
29
Neural
Back
end
Symbolic
Front
end

30. Learning intermediate abstractions
Faster adaptation to changes;
Better transfer learning

31. Learning intermediate abstractions
Faster adaptation to changes;
Better transfer learning

32. Injecting knowledge into rule-learning
32

33. 33
0
0

34. Injecting knowledge into rule-learning
• Learn the rules 𝑟𝑑and linear terms l()
• Learn the weights 𝛼 and 𝛽

35. Injecting knowledge into rule-learning
• Learn the rules 𝑟𝑑and linear terms l()
• Experts give the rules 𝑟𝑐 and 𝑟𝑐
• Learn the weights 𝛼 and 𝛽

36. Concluding
remarks
36

37. 1. Machine Learning can benefit from injecting
symbolic knowledge (knowledge graphs)
• Explanations
• Ranking hypotheses
• Transfer learning
• Sample efficiency
• Pipeline engineering
2. The required symbolic knowledge is available
at very large scale + corresponding tools
“Why learn what you already know?”
37