8. Lora Aroyo @laroyo http://lora-aroyo.org
chess
finite, mathematically well-defined search space
limited number of moves & states
grounded in explicit, unambiguous math rules
9. Lora Aroyo @laroyo http://lora-aroyo.org
IBM Confidential
scientific challenge in 2011
10. Lora Aroyo @laroyo http://lora-aroyo.org
IBM Confidential
natural language
ambiguous, contextual & implicit
grounded only in human cognition
infinite number of ways to express same meaning
11. Lora Aroyo @laroyo http://lora-aroyo.org
http://lora-aroyo.org @laroyo 11
keeps expanding to new domains
14. Lora Aroyo @laroyo http://lora-aroyo.org
1. Expand
“EXPANDs human
cognition - makes the jobs
we do easier, like a
cognitive prosthesis.
Especially when dealing
with processing massive
data, or data that requires
human interpretation”
15. Lora Aroyo @laroyo http://lora-aroyo.org
“LEARNs as you use it –
most machine errors are
easy for a human to
detect.
We can instrument usage
of systems to better
understand the system
and the problem it solves”
2. Learn
16. Lora Aroyo @laroyo http://lora-aroyo.org
“INTERACTs naturally -
bringing machines closer to
their users by letting them
understand natural language,
spoken or written, be able to
process images & videos.
These simple human problems
are extremely complex for
machines, but are hallmarks
of a new computing era.”
3. Interact
17. Lora Aroyo @laroyo http://lora-aroyo.org
▪ A new software paradigm has emerged
– Increasingly, computational tasks require inexact solutions that
combine multiple methods in unpredictable ways
▪ Knowledge is not the destination
– Watson does not answer a question by translating natural language
input into formally represented knowledge and simply running queries
against this knowledge
▪ Machine intelligence is not human intelligence
– The difference is most notable in the mistakes they make
When Building Cognitive Computing Systems …
http://www.yovisto.com/video/19674
Chris Welty’s keynote at WWW2012
18. Lora Aroyo @laroyo http://lora-aroyo.org
Cognitive
Systems
can fail
and still
be useful
A dramatic shift for software systems
Focus on improvement, not perfection
We must be able to answer, “Is it better?”
Cognitive Computing: It’s OK to be wrong!
19. Lora Aroyo @laroyo http://lora-aroyo.org
Cognitive Systems Engineering Pillars
MEASURE
MEASURE
MEASURE
DATA
DATA
DATA
TRUTH
TRUTH
TRUTH
20. Lora Aroyo @laroyo http://lora-aroyo.org
Cognitive Systems Engineering Pillars
MEASURE
MEASURE
MEASURE
DATA
DATA
DATA
TRUTH
TRUTH
TRUTH
They define the system.
If you change one of them,
the system should change
21. Lora Aroyo @laroyo http://lora-aroyo.org
Winning Human
Performance
2007 QA Computer System
Grand Champion
Human Performance
Top human
players are
remarkably
good.
Each dot – actual historical human Jeopardy! games
More Confident Less Confident
What it Takes to compete against Top Human Jeopardy! Players
Our Analysis Reveals the Winner’s Cloud
22. Lora Aroyo @laroyo http://lora-aroyo.org
2007 QA Computer System
In 2007, we committed to
making a Huge Leap!
Computers?
Not So Good.
Winning Human
Performance
Grand Champion
Human Performance
What it Takes to compete against Top Human Jeopardy! Players
Our Analysis Reveals the Winner’s Cloud
Each dot – actual historical human Jeopardy! games
More Confident Less Confident
23. Lora Aroyo @laroyo http://lora-aroyo.org
Precision
% of Answered
Deep QA: Incremental Progress
6/2007-11/2010
Now Playing in the
Winners Cloud
11/2010
04/2010
10/2009
5/2009
12/2008
8/2008
5/2008
12/2007
Baseline
25. Lora Aroyo @laroyo http://lora-aroyo.org
Watch…
Day 2 (15 Feb 2011)
Day 1 (14 Feb 2011)
Also on YouTube:
https://www.youtube.com/watch?v=i-vMW_Ce51w