Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Tetherless World Constellation, RPI
KR in the age of
Deep Learning,
Watson,
and the Semantic Web
Jim Hendler
Tetherless Wo...
Tetherless World Constellation, RPI
But first, Why the Moose?
This moose gave a keynote
with Tim Berners-Lee.
This moose g...
Tetherless World Constellation, RPI
Talk derives in large part from working on
forthcoming book
(More info at Springer boo...
Tetherless World Constellation, RPI
Outline
• Several important AI technologies have
moved through “knees in the curve”
br...
Tetherless World Constellation, RPI
A) Deep Learning
“phase transition” in capabilities of neural networks
w/machine power
Tetherless World Constellation, RPI
Trained on lots of categorized images
Imagenet: Duck Imagenet: Cat
Tetherless World Constellation, RPI
Impressive results
Increasingly powerful techniques have yielded
incredible results in...
Tetherless World Constellation, RPI
B) Watson
Tetherless World Constellation, RPI
The Watson DeepQA Pipeline
Tetherless World Constellation, RPI
Watson is based on ”Associative knowledge”
© IBM, used with permission.
Tetherless World Constellation, RPI
Impressive Results
Watson showed the power of “associative knowledge”
Tetherless World Constellation, RPI
C) Semantic Web
Tetherless World Constellation, RPI
From Semantic Web to the Knowledge Graph
Tetherless World Constellation, RPI
Based on a large “knowledge graph” mined from
extracted and learned data
Tetherless World Constellation, RPI
Many intermediate steps
(P. Norvig, WWW 2016, 4/16)
Tetherless World Constellation, RPI
Impressive results
Google finds embedded metadata on >30% of its crawl – Guha, 2015
Go...
Tetherless World Constellation, RPI
Summary: AI has done some way cool stuff
Summary (simplifying tremendously)
• Deep Lea...
Tetherless World Constellation, RPI
Why did knowledge graph need
“”Human Judgments”?
Association ≠ Correctness
Tetherless World Constellation, RPI
Quick quiz
Who did this moose give invited talks with?
A) Stuart Russell & Vint Cerf
B...
Tetherless World Constellation, RPI
Associational learning cannot
explain learning by “symbolic communication”
Who did thi...
Tetherless World Constellation, RPI
GOFAI: Knowledge Representation?
• A knowledge representation (KR) is most fundamental...
Tetherless World Constellation, RPI
KR: Human Expression
Cute kid story: first two words
Tetherless World Constellation, RPI
Telling cats from ducks doesn’t need KR
!
Tetherless World Constellation, RPI
“Saying things about the world” does
"If I was telling it to a
kid, I'd probably say
s...
Tetherless World Constellation, RPI
KR: Surrogate knowledge?
Which could you sit in?
What is most likely to bite what?
Whi...
Tetherless World Constellation, RPI
“Surrogate” knowledge
Which could you sit in?
What is most likely to bite what?
Which ...
Tetherless World Constellation, RPI
KR: Recommended vs. Possible inference
Which one would you save if the house was on fi...
Tetherless World Constellation, RPI
Recommended vs. Possible inference
Which one would you save if the house was on fire?
...
Tetherless World Constellation, RPI
KR systems in AI need grounded symbols
• Logic- and rule- based systems
– Ground in “m...
Tetherless World Constellation, RPI
The challenge
• If we want to implement KR systems
on top of neural and associative
le...
Tetherless World Constellation, RPI
The challenges
• Can we avoid throwing out the
reasoning baby with the grounding
bathw...
Tetherless World Constellation, RPI
Not just “theory” the applications driving
much modern AI require new grounding ideas
...
Tetherless World Constellation, RPI
Starting Place: Rethinking grounding
– Formal Explanation vs. post hoc
justification
•...
Tetherless World Constellation, RPI
Human-Aware AI
• Context is key
– AI systems still perform best in well-
defined conte...
Tetherless World Constellation, RPI
Why this REALLY matters
• Humanity faces huges challenges
– eg. Our knowledge of cance...
Tetherless World Constellation, RPI
Attacking these problems require the best minds we have working
together: Human and AI...
Tetherless World Constellation, RPI
Summary of talk (minus moose)
• Modern AI is making some huge strides
– Eg. DL, Associ...
Tetherless World Constellation, RPI
Questions?
Upcoming SlideShare
Loading in …5
×

Knowledge Representation in the Age of Deep Learning, Watson, and the Semantic Web

6,244 views

Published on

IJCAI 16 keynote on the need to bring modern AI accomplishments of recent years into connection with the more traditional goals of symbolic AI (and vice versa).

Published in: Technology
  • Be the first to comment

Knowledge Representation in the Age of Deep Learning, Watson, and the Semantic Web

  1. 1. Tetherless World Constellation, RPI KR in the age of Deep Learning, Watson, and the Semantic Web Jim Hendler Tetherless World Professor of Computer, Web and Cognitive Sciences Director, Institute for Data Exploration and Applications Rensselaer Polytechnic Institute http://www.cs.rpi.edu/~hendler @jahendler (twitter) Major talks at: http://www.slideshare.net/jahendler
  2. 2. Tetherless World Constellation, RPI But first, Why the Moose? This moose gave a keynote with Tim Berners-Lee. This moose gave a keynote with Peter Norvig.
  3. 3. Tetherless World Constellation, RPI Talk derives in large part from working on forthcoming book (More info at Springer booth) (Thanks Alice!)
  4. 4. Tetherless World Constellation, RPI Outline • Several important AI technologies have moved through “knees in the curve” bringing much of the attention to AI again – Deep Learning (& ML in general) – Watson (& “cognitive computing”) – Semantic Web (& the knowledge graph) • But what about KR – What it is, why it still matters • And how can these come together – Which comes with a lot of important challenges
  5. 5. Tetherless World Constellation, RPI A) Deep Learning “phase transition” in capabilities of neural networks w/machine power
  6. 6. Tetherless World Constellation, RPI Trained on lots of categorized images Imagenet: Duck Imagenet: Cat
  7. 7. Tetherless World Constellation, RPI Impressive results Increasingly powerful techniques have yielded incredible results in the past few years
  8. 8. Tetherless World Constellation, RPI B) Watson
  9. 9. Tetherless World Constellation, RPI The Watson DeepQA Pipeline
  10. 10. Tetherless World Constellation, RPI Watson is based on ”Associative knowledge” © IBM, used with permission.
  11. 11. Tetherless World Constellation, RPI Impressive Results Watson showed the power of “associative knowledge”
  12. 12. Tetherless World Constellation, RPI C) Semantic Web
  13. 13. Tetherless World Constellation, RPI From Semantic Web to the Knowledge Graph
  14. 14. Tetherless World Constellation, RPI Based on a large “knowledge graph” mined from extracted and learned data
  15. 15. Tetherless World Constellation, RPI Many intermediate steps (P. Norvig, WWW 2016, 4/16)
  16. 16. Tetherless World Constellation, RPI Impressive results Google finds embedded metadata on >30% of its crawl – Guha, 2015 Google “knowledge vault” reported to have over 1.6 billion “facts” (links)
  17. 17. Tetherless World Constellation, RPI Summary: AI has done some way cool stuff Summary (simplifying tremendously) • Deep Learning: neural learning from data with high quality, but imperfect results • Watson: Associative learning from data with high quality but imperfect results • Semantic Web/Knowledge Graph: Graph links formation from extraction, clustering and learning As much as many of us “GOFAI” folks wish it, this stuff cannot be ignored but, there are still problems…
  18. 18. Tetherless World Constellation, RPI Why did knowledge graph need “”Human Judgments”? Association ≠ Correctness
  19. 19. Tetherless World Constellation, RPI Quick quiz Who did this moose give invited talks with? A) Stuart Russell & Vint Cerf B) A deer and a keynote C) IJCAI-16 and Alces Alces D) Tim Berners-Lee and Peter Norvig
  20. 20. Tetherless World Constellation, RPI Associational learning cannot explain learning by “symbolic communication” Who did this moose give invited talks with? A) Stuart Russell & Vint Cerf (highly associated with target answer) B) A deer and a keynote (word embedding similarity to question) C) IJCAI-16 and Alces Alces (perceptually linked) D) Tim Berners-Lee and Peter Norvig (Correct answer is something most of you learned today, 1-shot, via being told)
  21. 21. Tetherless World Constellation, RPI GOFAI: Knowledge Representation? • A knowledge representation (KR) is most fundamentally a surrogate, a substitute for the thing itself, used to enable an entity to determine consequences by thinking rather than acting, i.e., by reasoning about the world rather than taking action in it. • It is a set of ontological commitments, i.e., an answer to the question: In what terms should I think about the world? • It is a fragmentary theory of intelligent reasoning, expressed in terms of three components: (i) the representation's fundamental conception of intelligent reasoning; (ii) the set of inferences the representation sanctions; and (iii) the set of inferences it recommends. • It is a medium for pragmatically efficient computation, i.e., the computational environment in which thinking is accomplished. One contribution to this pragmatic efficiency is supplied by the guidance a representation provides for organizing information so as to facilitate making the recommended inferences. • It is a medium of human expression, i.e., a language in which we say things about the world. R. Davis, H. Shrobe, P. Szolovits (1993)
  22. 22. Tetherless World Constellation, RPI KR: Human Expression Cute kid story: first two words
  23. 23. Tetherless World Constellation, RPI Telling cats from ducks doesn’t need KR !
  24. 24. Tetherless World Constellation, RPI “Saying things about the world” does "If I was telling it to a kid, I'd probably say something like 'the cat has fur and four legs and goes meow, the duck is a bird and it swims and goes quack’. " How would you explain the difference between a duck and a cat to a child? Woof
  25. 25. Tetherless World Constellation, RPI KR: Surrogate knowledge? Which could you sit in? What is most likely to bite what? Which one is most likely to become a computer scientist someday? …
  26. 26. Tetherless World Constellation, RPI “Surrogate” knowledge Which could you sit in? What is most likely to bite what? Which one is most likely to become a computer scientist someday? How would they go about doing it?
  27. 27. Tetherless World Constellation, RPI KR: Recommended vs. Possible inference Which one would you save if the house was on fire?
  28. 28. Tetherless World Constellation, RPI Recommended vs. Possible inference Which one would you save if the house was on fire? Would you use a robot baby-sitter without knowing which of the three possibilities it would choose?
  29. 29. Tetherless World Constellation, RPI KR systems in AI need grounded symbols • Logic- and rule- based systems – Ground in “model theory” with a notion of truth and falsity • Probabilistic Reasoning – P(A|B) requires A, B map to “meaningful” concepts, P to be a “real” probability • Constraint Satisfaction, etc – Finding an interpretation satisfying a set of boolean (T,F) constraints (Note: Yes, I am simplifying, blurring distinctions, ignoring much cutting edge work… happy to discuss later)
  30. 30. Tetherless World Constellation, RPI The challenge • If we want to implement KR systems on top of neural and associative learners we have an issue – The numbers coming out of Deep Learning and Associative graphs are not probabilities – They don’t necessarily ground in human-meaningful symbols • ”sub-symbolic” learning … • Association by clustering … • Errorful extraction …
  31. 31. Tetherless World Constellation, RPI The challenges • Can we avoid throwing out the reasoning baby with the grounding bathwater? – We still need planning systems – We still want to be able to define the rules that a system should follow – We want to be able to interact with and understand these systems • Even if computers don’t need to be symbolic communicators, WE DO!!!
  32. 32. Tetherless World Constellation, RPI Not just “theory” the applications driving much modern AI require new grounding ideas Guruduth Banavar, w/permission)
  33. 33. Tetherless World Constellation, RPI Starting Place: Rethinking grounding – Formal Explanation vs. post hoc justification • Eg. Even if we cannot use a formal decomposition to explain the reasoning, can we produce a justification that explains it – Reasoning systems that “know” some of their axioms may be simply wrong • Eg.F1 of .9 doesn’t mean answers are 90% correct, it is (simplifying) more like 9 out of 10 answers are right, the others aren’t. – Nailing context …
  34. 34. Tetherless World Constellation, RPI Human-Aware AI • Context is key – AI systems still perform best in well- defined contexts (or trained situations, or where their document set is complete, etc.) – Humans are good at recognizing context and deciding when extraneous factors don’t make sense • Extreme example: Stanislav Yevgrafovich Petrov (the man who saved the world)
  35. 35. Tetherless World Constellation, RPI Why this REALLY matters • Humanity faces huges challenges – eg. Our knowledge of cancer genomics is being outpaced by mutations as cancer continues to spread – eg. Our neighborhoods degrade as wealth disparity grows – eg. Our climate warms as we argue about the causes without changing behaviors
  36. 36. Tetherless World Constellation, RPI Attacking these problems require the best minds we have working together: Human and AI! The existential threat is not AI, it’s not utilizing the AI we have correctly
  37. 37. Tetherless World Constellation, RPI Summary of talk (minus moose) • Modern AI is making some huge strides – Eg. DL, Associative Learning, Knowledge Graphs, … • But the need for KR has not gone away – Eg. Surrogacy, Recommended Inference, Human communication • The integration challenge will require goring some sacred cows – Grounding, explanation, context …. • But we need to do it.
  38. 38. Tetherless World Constellation, RPI Questions?

×