Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Modular design patterns for
systems that learn and reason:
a boxology
Frank van Harmelen, Annette ten Teije (V1)
Vrije Uni...
Increasingly broad concensus in AI
(“the third wave”)
The next progress in AI will be driven by systems
that combine neura...
So let’s compare
3
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
10M training images
Symbolic Connectionist
Construction Human effort Data hunger
Scalable +/- +/-
E...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Strengths & Weaknesses
Symbolic Connectionist
Construction Human effort Data hunger
Scaleable +/- +/-
Explainable + -
Gene...
Can we get them
to collaborate?
12
So we started reading…
• 3 years of weekly reading group ≈ 75 papers
• 3x 8-week seminar with 15 students ≈100 papers
13
I...
14
Ontology learning
Description logic learning
Hyperbolic embeddings
15
Goal 1: Can we make a reading map?
(as educators)
Goal 2: Can we make a modular theory?
(as scientists)
Inspired by Software Engineering:
a theory of re-usable patterns
(“Gang of four”)
16
Inspired by Software Engineering:
a theory of re-usable patterns
17
Inspired by Process Mining:
a theory of re-usable patterns
18
Inspired by Knowledge Engineering:
a theory of re-usable patterns
19
knowledge-
intensive
task
analytic
task
classificatio...
Plan:
make compositional patterns
by loose coupling
of elementary components
20
a “boxology”
learning
inference
Example: a classical ML system
21
+
=
Example: Inductive Logic Programming
22
parent(Mary,Vicky).
parent(Mary,Andre).
parent(Carrey,Vickey).
mother(Mary,Vicky)....
Symbols in, symbols out
• Inductive Logic Programming
• Probabilistic Soft Logic
• Markov Logic Networks
• ….
23
Intermezzo: Symbol or data?
24
“A classical machine learning system: ”
“What the <0.70, 1.17, 0.99, 1.07> is a Symbol?”
Is...
Symbolic prior (informed ML)
P(cushion|chair) >> P(flower|chair)
25
See survey of 100+ systems in Von Rueden et al, Learni...
Learning intermediate abstractions
for reasoning
:- see( , 3), see( , 5), add(3,5,8).
End-to-end:
2x 784
inputs
19 outputs...
Learning intermediate abstractions
for learning
Neural
Back
end
Symbolic
Front
end
Example: Reinforcement learning for
spatial navigation
Faster adaptation to changes;
Better transfer learning
Explainable ML by rational
reconstruction
queen
crown
wears
29
shower
cap
Ranking hypotheses (≈ explaining why not)
queen
crown
wears
30
shower
cap
symbol
prediction
algorithm
From symbols to data and back again
Knowledge Graph completion
31
ML ML
From symbols to data and back again
Knowledge Graph completion
Rolling
Stones
Angi Beat It
Michael
Jackson
Publish_song
32...
prediction
algorithm
From symbols to data and back again
Knowledge Graph completion
33
ML ML
Knowledge-based auto-ML
• Algorithmic configuration
• Hyperparameter tuning
• Selection of
training examples
34
Concluding
remarks
35
Goal 1: Create some structure
in the huge number of proposals
for combining learning and reasoning
Goal 2: Create modular ...
Upcoming SlideShare
Loading in …5
×

of

Modular design patterns for systems that learn and reason: a boxology Slide 1 Modular design patterns for systems that learn and reason: a boxology Slide 2 Modular design patterns for systems that learn and reason: a boxology Slide 3 Modular design patterns for systems that learn and reason: a boxology Slide 4 Modular design patterns for systems that learn and reason: a boxology Slide 5 Modular design patterns for systems that learn and reason: a boxology Slide 6 Modular design patterns for systems that learn and reason: a boxology Slide 7 Modular design patterns for systems that learn and reason: a boxology Slide 8 Modular design patterns for systems that learn and reason: a boxology Slide 9 Modular design patterns for systems that learn and reason: a boxology Slide 10 Modular design patterns for systems that learn and reason: a boxology Slide 11 Modular design patterns for systems that learn and reason: a boxology Slide 12 Modular design patterns for systems that learn and reason: a boxology Slide 13 Modular design patterns for systems that learn and reason: a boxology Slide 14 Modular design patterns for systems that learn and reason: a boxology Slide 15 Modular design patterns for systems that learn and reason: a boxology Slide 16 Modular design patterns for systems that learn and reason: a boxology Slide 17 Modular design patterns for systems that learn and reason: a boxology Slide 18 Modular design patterns for systems that learn and reason: a boxology Slide 19 Modular design patterns for systems that learn and reason: a boxology Slide 20 Modular design patterns for systems that learn and reason: a boxology Slide 21 Modular design patterns for systems that learn and reason: a boxology Slide 22 Modular design patterns for systems that learn and reason: a boxology Slide 23 Modular design patterns for systems that learn and reason: a boxology Slide 24 Modular design patterns for systems that learn and reason: a boxology Slide 25 Modular design patterns for systems that learn and reason: a boxology Slide 26 Modular design patterns for systems that learn and reason: a boxology Slide 27 Modular design patterns for systems that learn and reason: a boxology Slide 28 Modular design patterns for systems that learn and reason: a boxology Slide 29 Modular design patterns for systems that learn and reason: a boxology Slide 30 Modular design patterns for systems that learn and reason: a boxology Slide 31 Modular design patterns for systems that learn and reason: a boxology Slide 32 Modular design patterns for systems that learn and reason: a boxology Slide 33 Modular design patterns for systems that learn and reason: a boxology Slide 34 Modular design patterns for systems that learn and reason: a boxology Slide 35 Modular design patterns for systems that learn and reason: a boxology Slide 36
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

2 Likes

Share

Download to read offline

Modular design patterns for systems that learn and reason: a boxology

Download to read offline

A set of modular design patterns that can describe a large number of neuro-symbolic architectures from the literature. Corresponding paper is at https://arxiv.org/abs/2102.11965

Related Books

Free with a 30 day trial from Scribd

See all

Modular design patterns for systems that learn and reason: a boxology

  1. 1. Modular design patterns for systems that learn and reason: a boxology Frank van Harmelen, Annette ten Teije (V1) Vrije Universiteit Amsterdam + Michael van Bekkum, Maaike de Boer, André Meyer (V2) TNO Netherlands (https://arxiv.org/abs/2102.11965) Creative Commons License CC BY 3.0: Allowed to copy, redistribute remix & transform But must attribute 1
  2. 2. Increasingly broad concensus in AI (“the third wave”) The next progress in AI will be driven by systems that combine neural and symbolic techniques Position papers by Marcus, Lamb & Garcez, Darwiche, Pearl, Kautz, … Keynotes at AAAI17, IJCAI18, IJCAI19, AAAI20,…. (“proof by authority”  ) 2
  3. 3. So let’s compare 3
  4. 4. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 4
  5. 5. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 300.000 medical definitions 40 years of effort, 10.000 updates every years 5 “knowledge acquisition bottle neck”
  6. 6. Strengths & Weaknesses 10M training images Symbolic Connectionist Construction Human effort Data hunger Scalable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 4.8M training games 6 “sample inefficiency”
  7. 7. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff worse with more data worse with less data 7 “sample inefficiency” “combinatorial explosion”
  8. 8. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 8 “black box problem”
  9. 9. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff quality  generality  9
  10. 10. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 10 Class: 793 Label: n04209133 (shower cap) Certainty: 99.7% “out of distribution generalisability”
  11. 11. Strengths & Weaknesses Symbolic Connectionist Construction Human effort Data hunger Scaleable +/- +/- Explainable + - Generalisable Performance cliff Performance cliff 11
  12. 12. Can we get them to collaborate? 12
  13. 13. So we started reading… • 3 years of weekly reading group ≈ 75 papers • 3x 8-week seminar with 15 students ≈100 papers 13 It was a mess… Lot’s of techniques, tricks, ideas, methods, math No structure, no guidance, no map, no theory
  14. 14. 14 Ontology learning Description logic learning Hyperbolic embeddings
  15. 15. 15 Goal 1: Can we make a reading map? (as educators) Goal 2: Can we make a modular theory? (as scientists)
  16. 16. Inspired by Software Engineering: a theory of re-usable patterns (“Gang of four”) 16
  17. 17. Inspired by Software Engineering: a theory of re-usable patterns 17
  18. 18. Inspired by Process Mining: a theory of re-usable patterns 18
  19. 19. Inspired by Knowledge Engineering: a theory of re-usable patterns 19 knowledge- intensive task analytic task classification synthetic task assessment diagnosis configuration design planning scheduling assignment modelling prediction monitoring design object class attribute feature truth value generate specify match obtain Task types Task templates
  20. 20. Plan: make compositional patterns by loose coupling of elementary components 20 a “boxology” learning inference
  21. 21. Example: a classical ML system 21 + =
  22. 22. Example: Inductive Logic Programming 22 parent(Mary,Vicky). parent(Mary,Andre). parent(Carrey,Vickey). mother(Mary,Vicky). mother(Mary,Andy). father(Carrey,Vickey). father(Carrey,Andy). parent(X,Y) :- mother(X,Y). parent(X,Y) :- father(X,Y) parent(Carrey,Andy).
  23. 23. Symbols in, symbols out • Inductive Logic Programming • Probabilistic Soft Logic • Markov Logic Networks • …. 23
  24. 24. Intermezzo: Symbol or data? 24 “A classical machine learning system: ” “What the <0.70, 1.17, 0.99, 1.07> is a Symbol?” Istvan Berkeley, Minds & Machines, 2008. 1. a symbol must designate an object, a class or a relation in the world (= the “interpretation” of the symbol) 2. symbols can be either atomic or complex, (= composed of other symbols according to compositional rules 3. there must be system of operations that, when applied to a symbol, generates new symbols, that again must have a designation. cat
  25. 25. Symbolic prior (informed ML) P(cushion|chair) >> P(flower|chair) 25 See survey of 100+ systems in Von Rueden et al, Learning, 2019 cushion
  26. 26. Learning intermediate abstractions for reasoning :- see( , 3), see( , 5), add(3,5,8). End-to-end: 2x 784 inputs 19 outputs AlphaGo
  27. 27. Learning intermediate abstractions for learning Neural Back end Symbolic Front end
  28. 28. Example: Reinforcement learning for spatial navigation Faster adaptation to changes; Better transfer learning
  29. 29. Explainable ML by rational reconstruction queen crown wears 29 shower cap
  30. 30. Ranking hypotheses (≈ explaining why not) queen crown wears 30 shower cap symbol
  31. 31. prediction algorithm From symbols to data and back again Knowledge Graph completion 31 ML ML
  32. 32. From symbols to data and back again Knowledge Graph completion Rolling Stones Angi Beat It Michael Jackson Publish_song 32 Angi Rolling Stones Publish_song From: Predict: ML ML
  33. 33. prediction algorithm From symbols to data and back again Knowledge Graph completion 33 ML ML
  34. 34. Knowledge-based auto-ML • Algorithmic configuration • Hyperparameter tuning • Selection of training examples 34
  35. 35. Concluding remarks 35
  36. 36. Goal 1: Create some structure in the huge number of proposals for combining learning and reasoning Goal 2: Create modular architectures Contribution: A set of re-usable architectural patterns for modular systems that learn and reason Next steps: • Formalise informal diagrams as pre/post-conditions • Implement informal diagrams as a code library • Generate diagrams via a grammar (and predict unexplored patterns) 36
  • ernestojimenezruiz

    Apr. 12, 2021
  • gmentzas

    Mar. 15, 2021

A set of modular design patterns that can describe a large number of neuro-symbolic architectures from the literature. Corresponding paper is at https://arxiv.org/abs/2102.11965

Views

Total views

433

On Slideshare

0

From embeds

0

Number of embeds

9

Actions

Downloads

14

Shares

0

Comments

0

Likes

2

×